The Apache Jakarta Tomcat 5.5 Servlet/JSP ContainerManager App HOW-TO | |
Introduction |
In many production environments, it is very useful to have the capability
to deploy a new web application, or undeploy an existing one, without having
to shut down and restart the entire container. In addition, you can request
an existing application to reload itself, even if you have not declared it
to be reloadable in the Tomcat 5 server
configuration file.
To support these capabilities, Tomcat 5 includes a web application
(installed by default on context path /manager ) that supports
the following functions:
- Deploy a new web application, on a specified context path, from
the uploaded contents of a WAR file.
- Deploy a new web application, on a specified context path, from the
server file system.
- List the currently deployed web applications, as well as the
sessions that are currently active for those web apps.
- Reload an existing web application, to reflect changes in the
contents of
/WEB-INF/classes or /WEB-INF/lib .
- List the OS and JVM property values.
- List the available global JNDI resources, for use in deployment
tools that are preparing
<ResourceLink> elements
nested in a <Context> deployment description.
- List the available security roles defined in the user database.
- Start a stopped application (thus making it available again).
- Stop an existing application (so that it becomes unavailable), but
do not undeploy it.
- Undeploy a deployed web application and delete its document base
directory (unless it was deployed from file system).
There are two ways to configure the Manager web application
Context :
If you have Tomcat configured to support multiple virtual hosts
(websites) you would need to configure a Manager for each.
There are three ways to use the Manager web application.
- As an application with a user interface you use in your browser.
Here is an example URL where you can replace
localhost with
your website host name: http://localhost/manager/html/ .
- A minimal version using HTTP requests only which is suitable for use
by scripts setup by system administrators. Commands are given as part of the
request URI, and responses are in the form of simple text that can be easily
parsed and processed. See
Supported Manager Commands for more information.
- A convenient set of task definitions for the Ant
(version 1.4 or later) build tool. See
Executing Manager Commands
With Ant for more information.
Future versions of Tomcat 5 will include administrative functionality that
is presented in (at least) the following forms:
- As web services, so that Tomcat administration can be easily integrated
into remote and/or non-Java mnagement environments.
- As a web application with a nice user interface (built on top of the
web services processing layer) for easy Tomcat administration via a
web browser.
|
Configuring Manager Application Access |
The description below uses the variable name $CATALINA_HOME
to refer to the directory into which you have installed Tomcat 5,
and is the base directory against which most relative paths are
resolved. However, if you have configured Tomcat 5 for multiple
instances by setting a CATALINA_BASE directory, you should use
$CATALINA_BASE instead of $CATALINA_HOME for each of these
references.
It would be quite unsafe to ship Tomcat with default settings that allowed
anyone on the Internet to execute the Manager application on your server.
Therefore, the Manager application is shipped with the requirement that anyone
who attempts to use it must authenticate themselves, using a username and
password that have the role manager associated with them.
Further, there is no username in the default users file
($CATALINA_HOME/conf/tomcat-users.xml) that is assigned this
role. Therefore, access to the Manager application is completely disabled
by default.
To enable access to the Manager web application, you must either create
a new username/password combination and associate the role name
manager with it, or add the manager role
to some existing username/password combination. Exactly where this is done
depends on which Realm implementation you are using:
- MemoryRealm - If you have not customized your
$CATALINA_HOME/conf/server.xml to select a different one,
Tomcat 5 defaults to an XML-format file stored at
$CATALINA_HOME/conf/tomcat-users.xml , which can be
edited with any text editor. This file contains an XML
<user> for each individual user, which might
look something like this:
| | | |
<user name="craigmcc" password="secret" roles="standard,manager" />
| | | | |
which defines the username and password used by this individual to
log on, and the role names he or she is associated with. You can
add the manager role to the comma-delimited
roles attribute for one or more existing users, and/or
create new users with that assigned role.
- JDBCRealm - Your user and role information is stored in
a database accessed via JDBC. Add the manager role
to one or more existing users, and/or create one or more new users
with this role assigned, following the standard procedures for your
environment.
- JNDIRealm - Your user and role information is stored in
a directory server accessed via LDAP. Add the manager
role to one or more existing users, and/or create one or more new users
with this role assigned, following the standard procedures for your
environment.
The first time you attempt to issue one of the Manager commands
described in the next section, you will be challenged to log on using
BASIC authentication. The username and password you enter do not matter,
as long as they identify a valid user in the users database who possesses
the role manager.
In addition to the password restrictions the manager web application
could be restricted by the remote IP address or host by adding a
RemoteAddrValve or RemoteHostValve . Here is
an example of restricting access to the localhost by IP address:
<Context path="/manager" debug="0" privileged="true"
docBase="/usr/local/kinetic/tomcat5/server/webapps/manager">
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
allow="127.0.0.1"/>
</Context>
|
Supported Manager Commands |
All commands that the Manager application knows how to process are
specified in a single request URI like this:
| | | |
http://{host}:{port}/manager/{command}?{parameters}
| | | | |
where {host} and {port} represent the hostname
and port number on which Tomcat is running, {command}
represents the Manager command you wish to execute, and
{parameters} represents the query parameters
that are specific to that command. In the illustrations below, customize
the host and port appropriately for your installation.
Most commands accept one or more of the following query parameters:
- path - The context path (including the leading slash)
of the web application you are dealing with. To select the ROOT web
application, specify "/". NOTE -
It is not possible to perform administrative commands on the
Manager application itself.
- war - URL of a web application archive (WAR) file,
pathname of a directory which contains the web application, or a
Context configuration ".xml" file. You can use URLs in any of the
following formats:
- file:/absolute/path/to/a/directory - The absolute
path of a directory that contains the unpacked version of a web
application. This directory will be attached to the context path
you specify without any changes.
- file:/absolute/path/to/a/webapp.war - The absolute
path of a web application archive (WAR) file. This is valid
only for the
/deploy command, and is
the only acceptable format to that command.
- jar:file:/absolute/path/to/a/warfile.war!/ - The
URL to a local web application archive (WAR) file. You can use any
syntax that is valid for the
JarURLConnection class
for reference to an entire JAR file.
- file:/absolute/path/to/a/context.xml - The
absolute path of a web application Context configuration ".xml"
file which contains the Context configuration element.
- directory - The directory name for the web
applciation context in the Host's application base directory.
- webapp.war - The name of a web application war file
located in the Host's application base directory.
Each command will return a response in text/plain format
(i.e. plain ASCII with no HTML markup), making it easy for both humans and
programs to read). The first line of the response wil begin with either
OK or FAIL , indicating whether the requested
command was successful or not. In the case of failure, the rest of the first
line will contain a description of the problem that was encountered. Some
commands include additional lines of information as described below.
Internationalization Note - The Manager application looks up
its message strings in resource bundles, so it is possible that the strings
have been translated for your platform. The examples below show the English
version of the messages.
WARNING: the legacy commands /install and
/remove are deprecated.
They are presently equivalent to /deploy and /undeploy ,
but could be removed in a future release.
Deploy A New Application Remotely |
| | | |
http://localhost:8080/manager/deploy?path=/foo
| | | | |
Upload the web application archive (WAR) file that is specified as the
request data in this HTTP PUT request, install it into the appBase
directory of our corresponding virtual host, and start it on the context path
specified by the path request parameter. If no path
is specified the directory name or the war file name without the .war extension
is used as the path. The application can
later be undeployed (and the corresponding application directory removed)
by use of the /undeploy .
The .WAR file may include Tomcat specific deployment configuration, by
including a Context configuration XML file in
/META-INF/context.xml .
URL parameters include:
update : When set to true, any existing update will be
undeployed first. The default value is set to false.
tag : Specifying a tag name, this allows associating the
deployed webapp with a version number. The application version can
be later redeployed when needed using only the tag.
NOTE - This command is the logical
opposite of the /undeploy command.
If installation and startup is successful, you will receive a response
like this:
| | | |
OK - Deployed application at context path /foo
| | | | |
Otherwise, the response will start with FAIL and include an
error message. Possible causes for problems include:
- Application already exists at path /foo
The context paths for all currently running web applications must be
unique. Therefore, you must undeploy the existing web
application using this context path, or choose a different context path
for the new one. The update parameter may be specified as
a parameter on the URL, with a value of true to avoid this
error. In that case, an undeploy will be performed on an existing
application before performing the deployment.
- Encountered exception
An exception was encountered trying to start the new web application.
Check the Tomcat 5 logs for the details, but likely explanations include
problems parsing your /WEB-INF/web.xml file, or missing
classes encountered when initializing application event listeners and
filters.
- Invalid context path was specified
The context path must start with a slash character. To reference the
ROOT web application use "/".
- No context path was specified
The path parameter is required.
|
Deploy A New Application from a Local Path |
Deploy and start a new web application, attached to the specified context
path (which must not be in use by any other web application).
This command is the logical opposite of the /undeploy command.
There are a number of different ways the deploy command can be used.
Deploy a version of a previously deployed webapp
This can be used to deploy a previous version of a web application, which
has been deployed using the tag attribute. Note that the work
directory for the manager webapp will contain the previously deployed WARs;
removing it would make the deployment fail.
| | | |
http://localhost:8080/manager/deploy?path=/footoo&tag=footag
| | | | |
Deploy a Directory or WAR by URL
Deploy a web application directory or ".war" file located on the Tomcat
server. If no path is specified, the directory name or the war file
name without the ".war" extension is used as the path. The war
parameter specifies a URL (including the file: scheme) for either
a directory or a web application archive (WAR) file. The supported syntax for
a URL referring to a WAR file is described on the Javadocs page for the
java.net.JarURLConnection class. Use only URLs that refer to
the entire WAR file.
In this example the web application located in the directory
/path/to/foo on the Tomcat server is deployed as the
web application context named /footoo .
| | | |
http://localhost:8080/manager/deploy?path=/footoo&war=file:/path/to/foo
| | | | |
In this example the ".war" file /path/to/bar.war on the
Tomcat server is deployed as the web application context named
/bar . Notice that there is no path parameter
so the context path defaults to the name of the web application archive
file without the ".war" extension.
| | | |
http://localhost:8080/manager/deploy?war=jar:file:/path/to/bar.war!/
| | | | |
Deploy a Directory or War from the Host appBase
Deploy a web application directory or ".war" file located in your Host
appBase directory. If no path is specified the directory name
or the war file name without the ".war" extension is used as the path.
In this example the web application located in a sub directory named
foo in the Host appBase directory of the Tomcat server is
deployed as the web application context named /foo . Notice
that there is no path parameter so the context path defaults
to the name of the web application directory.
| | | |
http://localhost:8080/manager/deploy?war=foo
| | | | |
In this example the ".war" file bar.war located in your
Host appBase directory on the Tomcat server is deployed as the web
application context named /bartoo .
| | | |
http://localhost:8080/manager/deploy?path=/bartoo&war=bar.war
| | | | |
Deploy using a Context configuration ".xml" file
If the Host deployXML flag is set to true you can deploy a web
application using a Context configuration ".xml" file and an optional
".war" file or web application directory. The context path
is not used when deploying a web application using a context ".xml"
configuration file.
A Context configuration ".xml" file can contain valid XML for a
web application Context just as if it were configured in your
Tomcat server.xml configuration file. Here is an
example:
| | | |
<Context path="/foobar" docBase="/path/to/application/foobar"
debug="0">
<!-- Link to the user database we will get roles from -->
<ResourceLink name="users" global="UserDatabase"
type="org.apache.catalina.UserDatabase"/>
</Context>
| | | | |
When the optional war parameter is set to the URL
for a web application ".war" file or directory it overrides any
docBase configured in the context configuration ".xml" file.
Here is an example of deploying an application using a Context
configuration ".xml" file.
| | | |
http://localhost:8080/manager/deploy?config=file:/path/context.xml
| | | | |
Here is an example of deploying an application using a Context
configuration ".xml" file and a web application ".war" file located
on the server.
| | | |
http://localhost:8080/manager/deploy?config=file:/path/context.xml&war=jar:file:/path/bar.war!/
| | | | |
Deployment Notes
If the Host is configured with unpackWARs=true and you deploy a war
file, the war will be unpacked into a directory in your Host appBase
directory.
If the application war or directory is installed in your Host appBase
directory and either the Host is configured with autoDeploy=true or
liveDeploy=true, the Context path must match the directory name or
war file name without the ".war" extension.
For security when untrusted users can manage web applications, the
Host deployXML flag can be set to false. This prevents untrusted users
from deploying web applications using a configuration XML file and
also prevents them from deploying application directories or ".war"
files located outside of their Host appBase.
Deploy Response
If installation and startup is successful, you will receive a response
like this:
| | | |
OK - Deployed application at context path /foo
| | | | |
Otherwise, the response will start with FAIL and include an
error message. Possible causes for problems include:
- Application already exists at path /foo
The context paths for all currently running web applications must be
unique. Therefore, you must undeploy the existing web
application using this context path, or choose a different context path
for the new one. The update parameter may be specified as
a parameter on the URL, with a value of true to avoid this
error. In that case, an undeploy will be performed on an existing
application before performing the deployment.
- Document base does not exist or is not a readable directory
The URL specified by the war parameter must identify a
directory on this server that contains the "unpacked" version of a
web application, or the absolute URL of a web application archive (WAR)
file that contains this application. Correct the value specified by
the war parameter.
- Encountered exception
An exception was encountered trying to start the new web application.
Check the Tomcat 5 logs for the details, but likely explanations include
problems parsing your /WEB-INF/web.xml file, or missing
classes encountered when initializing application event listeners and
filters.
- Invalid application URL was specified
The URL for the directory or web application that you specified
was not valid. Such URLs must start with file: , and URLs
for a WAR file must end in ".war".
- Invalid context path was specified
The context path must start with a slash character. To reference the
ROOT web application use "/".
- Context path must match the directory or WAR file name:
If the application war or directory is installed in your Host appBase
directory and either the Host is configured with autoDeploy=true or
liveDeploy=true, the Context path must match the directory name or
war file name without the ".war" extension.
- Only web applications in the Host web application directory can
be installed
If the Host deployXML flag is set to false this error will happen
if an attempt is made to deploy a web application directory or
".war" file outside of the Host appBase directory.
|
List Available Global JNDI Resources |
| | | |
http://localhost:8080/manager/resources[?type=xxxxx]
| | | | |
List the global JNDI resources that are available for use in resource
links for context configuration files. If you specify the type
request parameter, the value must be the fully qualified Java class name of
the resource type you are interested in (for example, you would specify
javax.sql.DataSource to acquire the names of all available
JDBC data sources). If you do not specify the type request
parameter, resources of all types will be returned.
Depending on whether the type request parameter is specfied
or not, the first line of a normal response will be:
OK - Listed global resources of all types
or
OK - Listed global resources of type xxxxx
followed by one line for each resource. Each line is composed of fields
delimited by colon characters (":"), as follows:
- Global Resource Name - The name of this global JNDI resource,
which would be used in the
global attribute of a
<ResourceLink> element.
- Global Resource Type - The fully qualified Java class name of
this global JNDI resource.
If an error occurs, the response will start with FAIL and
include an error message. Possible causes for problems include:
- Encountered exception
An exception was encountered trying to enumerate the global JNDI
resources. Check the Tomcat 5 logs for the details.
- No global JNDI resources are available
The Tomcat server you are running has been configured without
global JNDI resources.
|
|
Executing Manager Commands With Ant |
In addition to the ability to execute Manager commands via HTTP requests,
as documented above, Tomcat 5 includes a convenient set of Task definitions
for the Ant (version 1.4 or later) build tool. In order to use these
commands, you must perform the following setup operations:
- Download the binary distribution of Ant from
http://ant.apache.org.
You must use version 1.4 or later.
- Install the Ant distribution in a convenient directory (called
ANT_HOME in the remainder of these instructions).
- Copy the file
server/lib/catalina-ant.jar from your Tomcat 5
installation into Ant's library directory ($ANT_HOME/lib ).
- Add the
$ANT_HOME/bin directory to your PATH
environment variable.
- Configure at least one username/password combination in your Tomcat
user database that includes the
manager role.
To use custom tasks within Ant, you must declare them first with a
<taskdef> element. Therefore, your build.xml
file might look something like this:
<project name="My Application" default="compile" basedir=".">
<!-- Configure the directory into which the web application is built -->
<property name="build" value="${basedir}/build"/>
<!-- Configure the context path for this application -->
<property name="path" value="/myapp"/>
<!-- Configure properties to access the Manager application -->
<property name="url" value="http://localhost:8080/manager"/>
<property name="username" value="myusername"/>
<property name="password" value="mypassword"/>
<!-- Configure the custom Ant tasks for the Manager application -->
<taskdef name="deploy" classname="org.apache.catalina.ant.DeployTask"/>
<taskdef name="list" classname="org.apache.catalina.ant.ListTask"/>
<taskdef name="reload" classname="org.apache.catalina.ant.ReloadTask"/>
<taskdef name="resources" classname="org.apache.catalina.ant.ResourcesTask"/>
<taskdef name="roles" classname="org.apache.catalina.ant.RolesTask"/>
<taskdef name="start" classname="org.apache.catalina.ant.StartTask"/>
<taskdef name="stop" classname="org.apache.catalina.ant.StopTask"/>
<taskdef name="undeploy" classname="org.apache.catalina.ant.UndeployTask"/>
<!-- Executable Targets -->
<target name="compile" description="Compile web application">
<!-- ... construct web application in ${build} subdirectory, and
generated a ${path}.war ... -->
</target>
<target name="deploy" description="Install web application"
depends="compile">
<deploy url="${url}" username="${username}" password="${password}"
path="${path}" war="${build}${path}.war"/>
</target>
<target name="reload" description="Reload web application"
depends="compile">
<reload url="${url}" username="${username}" password="${password}"
path="${path}"/>
</target>
<target name="undeploy" description="Remove web application">
<undeploy url="${url}" username="${username}" password="${password}"
path="${path}"/>
</target>
</project>
|
Now, you can execute commands like ant deploy to deploy the
application to a running instance of Tomcat, or ant reload to
tell Tomcat to reload it. Note also that most of the interesting values in
this build.xml file are defined as replaceable properties, so
you can override their values from the command line. For example, you might
consider it a security risk to include the real manager password in your
build.xml file's source code. To avoid this, omit the password
property, and specify it from the command line:
ant -Dpassword=secret deploy
Tasks output capture |
Using Ant version 1.6.2 or later,
the Catalina tasks offer the option to capture their output in
properties or external files. They support directly the following subset of the
<redirector> type attributes:
Attribute |
Description |
Required |
output |
Name of a file to which to write the output. If
the error stream is not also redirected to a file or property, it will
appear in this output. |
No |
error |
The file to which the standard error of the
command should be redirected. |
No |
logError |
This attribute is used when you wish to see
error output in Ant's log and you are redirecting output to a
file/property. The error output will not be included in the output
file/property. If you redirect error with the error or errorProperty
attributes, this will have no effect. |
No |
append |
Whether output and error files should be
appended to or overwritten. Defaults to false . |
No |
createemptyfiles |
Whether output and error files should be created
even when empty. Defaults to true . |
No |
outputproperty |
The name of a property in which the output of
the command should be stored. Unless the error stream is redirected to
a separate file or stream, this property will include the error output. |
No |
errorproperty |
The name of a property in which the standard
error of the command should be stored. |
No |
A couple of additional attributes can also be specified:
Attribute |
Description |
Required |
alwaysLog |
This attribute is used when you wish to see the
output you are capturing, appearing also in the Ant's log. It must not be
used unless you are capturing task output.
Defaults to false .
This attribute will be supported directly by <redirector>
in Ant 1.6.3 |
No |
failonerror |
This attribute is used when you wish to avoid that
any manager command processing error terminates the ant execution. Defaults to true .
It must be set to false , if you want to capture error output,
otherwise execution will terminate before anything can be captured.
This attribute acts only on manager command execution,
any wrong or missing command attribute will still cause Ant execution termination.
|
No |
They also support the embedded <redirector> element
in which you can specify
its full set of attributes, but input , inputstring and
inputencoding that, even if accepted, are not used because they have
no meaning in this context.
Refer to ant manual for details on
<redirector> element attributes.
Here is a sample build file extract that shows how this output redirection support
can be used:
<target name="manager.deploy"
depends="context.status"
if="context.notInstalled">
<deploy url="${mgr.url}"
username="${mgr.username}"
password="${mgr.password}"
path="${mgr.context.path}"
config="${mgr.context.descriptor}"/>
</target>
<target name="manager.deploy.war"
depends="context.status"
if="context.deployable">
<deploy url="${mgr.url}"
username="${mgr.username}"
password="${mgr.password}"
update="${mgr.update}"
path="${mgr.context.path}"
war="${mgr.war.file}"/>
</target>
<target name="context.status">
<property name="running" value="${mgr.context.path}:running"/>
<property name="stopped" value="${mgr.context.path}:stopped"/>
<list url="${mgr.url}"
outputproperty="ctx.status"
username="${mgr.username}"
password="${mgr.password}">
</list>
<condition property="context.running">
<contains string="${ctx.status}" substring="${running}"/>
</condition>
<condition property="context.stopped">
<contains string="${ctx.status}" substring="${stopped}"/>
</condition>
<condition property="context.notInstalled">
<and>
<isfalse value="${context.running}"/>
<isfalse value="${context.stopped}"/>
</and>
</condition>
<condition property="context.deployable">
<or>
<istrue value="${context.notInstalled}"/>
<and>
<istrue value="${context.running}"/>
<istrue value="${mgr.update}"/>
</and>
<and>
<istrue value="${context.stopped}"/>
<istrue value="${mgr.update}"/>
</and>
</or>
</condition>
<condition property="context.undeployable">
<or>
<istrue value="${context.running}"/>
<istrue value="${context.stopped}"/>
</or>
</condition>
</target>
|
WARNING: even if it doesn't make many sense, and is always a bad idea,
calling a Catalina task more than once,
badly set Ant tasks depends chains may cause that a task be called
more than once in the same Ant run, even if not intended to. A bit of caution should be exercised when you are
capturing output from that task, because this could lead to something unexpected:
- when capturing in a property you will find in it only the output from the first call, because
Ant properties are immutable and once set they cannot be changed,
- when capturing in a file, each run will overwrite it and you will find in it only the last call
output, unless you are using the
append="true" attribute, in which case you will
see the output of each task call appended to the file.
|
|
Using the JMX Proxy Servlet |
What is JMX Proxy Servlet |
The JMX Proxy Servlet is a lightweight proxy to get and set the
tomcat internals. (Or any class that has been exposed via an MBean)
Its usage is not very user friendly but the UI is
extremely help for integrating command line scripts for monitoring
and changing the internals of tomcat. You can do two things with the proxy:
get information and set information. For you to really understand the
JMX Proxy Servlet, you should have a general understanding of JMX.
If you don't know what JMX is, then prepare to be confused.
|
JMX Query command |
This takes the form:
| | | |
http://webserver/manager/jmxproxy/?qry=STUFF
| | | | |
Where STUFF is the JMX query you wish to perform. For example,
here are some queries you might wish to run:
-
qry=*%3Atype%3DRequestProcessor%2C* -->
type=RequestProcessor which will locate all
workers which can process requests and report
their state.
-
qry=*%3Aj2eeType=Servlet%2c* -->
j2eeType=Servlet which return all loaded servlets.
-
qry=Catalina%3Atype%3DEnvironment%2Cresourcetype%3DGlobal%2Cname%3DsimpleValue -->
Catalina:type=Environment,resourcetype=Global,name=simpleValue
which look for a specific MBean by the given name.
You'll need to experiment with this to really understand its capabilites.
If you provide no qry parameter, then all of the MBeans will
be displayed. We really recommend looking at the tomcat source code and
understand the JMX spec to get a better understanding of all the queries
you may run.
|
JMX Set command |
Now that you can query an MBean, its time to muck with Tomcat's internals!
The general form of the set command is :
| | | |
http://webserver/manager/jmxproxy/?set=BEANNAME&att=MYATTRIBUTE&val=NEWVALUE
| | | | |
So you need to provide 3 request parameters:
set : The full bean name
att : The attribute you wish to alter
val : The new value
If all goes ok, then it will say OK, otherwise an error message will be
shown. For example, lets say we wish to turn up debugging on the fly for the
ErrorReportValve . The following will set debugging to 10.
| | | |
http://localhost:8080/manager/jmxproxy/
?set=Catalina%3Atype%3DValve%2Cname%3DErrorReportValve%2Chost%3Dlocalhost&att=debug&val=10
| | | | |
and my result is (YMMV):
Here is what I see if I pass in a bad value. Here is the URL I used,
I try set debugging equal to 'cowbell':
| | | |
http://localhost:8080/manager/jmxproxy/
?set=Catalina%3Atype%3DValve%2Cname%3DErrorReportValve%2Chost%3Dlocalhost&att=debug&val=cowbell
| | | | |
When I try that, my result is
| | | |
Error: java.lang.NumberFormatException: For input string: "cowbell"
| | | | |
|
|
|