Monday, April 16, 2012

Troubleshooting with JBoss 7

Recently I coming across a lot of configurations that are required for new servers like JBoss 6 & 7. So I thought of jotting them at a single place. I may be mentioning them in brief sometimes and sometimes I would provide as much detail as I can.

  • JBoss & Cookie Expiry : With JBoss 6 and 7, cookies by default specify the expiration time of cookie as "Max-Age" header attribute instead of "Expires". I have written a detail about this here.
  • Random number generation and server start up is slow on Unix platforms for some of the servers. This is because of /dev/random is used in Unix platforms for random number generation. I have added a detail entry explaining the problem and solution here.
  • JBoss 7 AS, has modular class loading. It provides true application isolation, hiding server implementation classes from application and only loading the classes your application needs. Modules, packaged as collections of classes, are peers that remain isolated unless explicitly defined as a dependency of another module. These visibility rules  can be customized. 

    Due to this design, JBoss modules does not load everything from rt.jar and many other system jars by default. As a result you may get ClassNotFoundException or NoClassDefFounfError for a lot of classes. Read here for solution and workarounds available for this.
  • When running Jsafe library on JBoss, if you get error like java.lang.NoClassDefFoundError put the jsafe (CryptoJ) jars to <JAVA_HOME>/jre/lib/ext directory. This mostly occurs on Unix platforms.
  • In JBoss version 7 or above if open office is not working start JBoss with flag -Djava.ext.dirs=<jboss-deployment-dir>/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib/oosdk/:<java-home>/lib/ext/ .

I will keep updating this space as and when i will find more.

JBoss 7 & Sandbox

JBoss 7 AS, has modular class loading. It provides true application isolation, hiding server implementation classes from application and only loading the classes your application needs. Modules, packaged as collections of classes, are peers that remain isolated unless explicitly defined as a dependency of another module. These visibility rules  can be customized. 

Due to this design, JBoss modules does not load everything from rt.jar and many other system jars by default. As a result you may get ClassNotFoundException or NoClassDefFounfError for a lot of classes. 

Example: 
com.sun.imageio.*
javax.annotation.processing.*
com.aqua.LookAndFeel
Open Office, web services tests might fail

Solution: jboss-deployment-structure.xml  -

To Fix this the solution is to, use a jboss-deployment-structure.xml in your war as explained here https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7

Example:

<jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.1">
    <deployment>
        <dependencies>
         <module name="deployment.system"/>
        </dependencies>
    </deployment>
    <module name="deployment.system">
     <dependencies>
         <system export="true">
             <paths>
                 <path name="com/sun/net/ssl/internal/ssl"/>
             </paths>
         </system>
     </dependencies>
    </module>
</jboss-deployment-structure>


However if for some reason you can't add jboss-deployment-structure.xml quickly in your application following are 2 workarounds.

Workarounds 1: For JRE System classes, There is a modue named "sun.jdk" that depdends on module "system". Here are defined all packages jboss loads from rt.jar.

In this module.xml of /sun/jdk/main directory add the required packages as -
                        <path name="com/sun/imageio/spi"/>
                        <path name="javax/annotation/processing"/>
etc, according to the need of application.

Example:
<module xmlns="urn:jboss:module:1.0" name="sun.jdk">
    <resources>
        <resource-root path="service-loader-resources"/>
    </resources>
    <dependencies>
        <module name="system" export="false" services="import">
            <exports>
                <include-set>

                    <!-- extra packages -->
                    <path name="javax/annotation/processing"/>

                    <path name="com/sun/script/javascript"/>
                    <path name="com/sun/imageio/spi"/>
                    <path name="com/sun/jndi/dns"/>
                    <path name="com/sun/jndi/ldap"/>
                    <path name="com/sun/security/auth"/>
                    <path name="com/sun/security/auth/module"/>
                    <path name="sun/misc"/>
                    <path name="sun/nio"/>
                    <path name="sun/nio/ch"/>
                    <path name="sun/util"/>
                    <path name="sun/util/calendar"/>
                    <path name="META-INF/services"/>
                </include-set>
            </exports>
        </module>
    </dependencies>
</module>

Workaround 2:

Add jboss.modules.system.pkgs system property with comma separated list of packages to load. 

-Djboss.modules.system.pkgs=com.sun.imageio.spi,javax.annotation.processing

Additional Information:

Related issues - 

JBoss cfusion.ear/war deployment timeout:

If cfusion.ear is not deployed within the deployment timeout for jboss you can change the deployment timeout of jboss in standalone.xml
<subsystem xmlns="urn:jboss:domain:deployment-scanner:1.0">
   <deployment-scanner scan-interval="5000"
      relative-to="jboss.server.base.dir" path="deployments"   deployment-timeout=”120”,/>
</subsystem>

Profiling:

Performance tuning is an important part any application development.  In order to use JProfiler ofor jboss 7, first you need to add the following to your standalone.conf file:

JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=com.jprofiler -agentlib:jprofilerti -Xbootclasspath/a:/path/to/jprofiler/bin/agent.jar"

The "jboss.modules.system.pkgs" property tells JBoss Modules to allow the "com.profiler" classes to be found from any class loader, which is essential to allow the JProfiler agent to run. It is easier to then just set up your jprofiler sessio nas "Remote", and start the server and profiler.

Friday, April 13, 2012

Random number generation in Unix & Performance


Random number generation and server start up is slow on Unix platforms for some of the servers. This is because of /dev/random is used in Unix platforms for random number generation. 


I am including an explanation and solution here. Feel free to go to Solution directly in case you understand the problem already. 


java.security.SecureRandom is designed to be crypto secure. It provides strong and secure random numbers. SecureRandom should be used when high-quality randomness is important and is worth consuming CPU. 

SecureRandom uses OS provided entropy for generating strong random numbers. Depending on the machine and environment, sources of entropy varies. The Operating system knows how and where to collect the entropy from. So it collects this entropy and makes it available via an API like CryptGenRandom() on windows and by reading from /dev/random device file for Unix like systems e.g. Linux, Solaris, Mac.

To obtain a series of bytes of entropy, you can call SecureRandom.getSeed(), with an instance of SecureRandom.  For Oracle Java, SecureRandom.generateSeed() is a wrapper for  os provided source of entropy, if one is available.

For Unix platforms /dev/random this entropy is generating by recording user actions with devices such as mouse clicks, key board strokes,  arriaval/access to disk/network packets etc.

While using SecureRandom, you should keep following in mind: 
  • in the worst case, the call SecureRandom.generateSeed() may block until the required entropy has been generated.
  • you should consider bits of entropy as a "valuable, shared resource"— if you request entropy faster than it is generated, then your application will block, and so potentially will other applicatons also requesting entropy.


This is exactly the problem that causes these servers to load slow. The alternative to this is /dev/urandom.


The difference between /dev/random and /dev/urandom is that 
/dev/random provides a limited (but relatively large) number of random bytes, and will block waiting that the kernel gives some more if the buffer is outrun, 


while /dev/urandom will always provide random bytes though they may become of a lesser quality once the initial buffer is outrun. However note that I am not implying that it is not secure at all.

Before Java 1.5, you could tell java to use /dev/urandom in case the application has higher need of performance. However  Java 1.5 onward, using "/dev/urandom" is ignored  and /dev/urandom file is mapped to /dev/random. See Bug 6202721
Solutions:
There are some solutions mentioned on the internet. I will list all those and then will highlight the one which i found working perfect in all situations -
  1. Leave it as is where /dev/random is used (even if set to /dev/urandom) and use some third party tool to introduce sufficient random entropy into your system so /dev/random doesn't block so slowly.
  2. Add  “-Djava.security.egd=file:/dev/./urandom” (/dev/urandom does not work) to java parameters.
  3. mv /dev/random /dev/random.ORIG ; ln /dev/urandom /dev/random
  4. Change $JAVA_HOME/jre/lib/security/java.security. Replace securerandom.source with securerandom.source=file:/dev/./urandom.
Note: #1 is the best solution and most secure. However for development phase and test environments (even for systems where you are ready to compromise on security), #4 solution is what I used and I recommend. 


Hope this explains and saves some time for you all. 
Have fun.



Wednesday, April 11, 2012

ColdFusion 10 & Secure Profile



How many times Administrators might have wondered what all to configure to make the server release ready and secure. There are different guidelines and recommendation documents out there. 


With ColdFusion 10, option to have Secure Profile enabled is added at installation time. It is recommended to choose this for production or public facing servers. When selected, this will enforce a lot of security related configurations enabling fewer configurations required by administrator to secure the server. 


At time of installation if secure profile is chosen, following settings are affected –

1.    Separate username and password setting is enabled for Administrator &RDS
2.    RDS service is disabled
3.    A List of IP addresses are asked which should have access to Administrator
4.    Strong and complex password for root admin user is mandatory
5.    Directory Browsing is disable in server
6.    Custom and least information error templates are used
7.    All debugging is disabled
8.    For a new data source default allowed SQL are select, insert, update, delete
9.    Below is a list of some more settings on server level which are affected -

Administrator settings affected by enabling Secure Profile -

Administrator Settings
Path
Default Admin
Profile
Secure Profile
Changes to the setting
1
Use UUID for cftoken
Server Settings > Settings
Enabled
Enabled
Overwritten
2
Disable access to internal
ColdFusion Java components
Server Settings > Settings
Disabled
Enabled
Overwritten
3
Enable Global Script
Protection
Server Settings > Settings
Enabled
Enabled
Overwritten
4
Maximum size of post data
Server Settings > Settings
20MB
20MB
Overwritten
5
Missing Template Handler
Server Settings > Settings
no value
Custom missing error template
Retained if specified
6
Site-wide Error Handler
Server Settings > Settings
no value
Custom site-wide error template
Retained if specified
7
Request Queue Timeout Page
Server Settings > Request
Tuning
no value
Custom error template
Retained if specified
8
Cookie Timeout
Server Settings > Memory
Variables
15767000 minute
1440 minute
N/A
9
Disabling updating of ColdFusion internal cookies using ColdFusion tags/functions
Server Settings > Memory
Variables
Disabled
Enabled
N/A
10
Enabled WebSocket Server
Server Settings > WebSocket
Enabled
Disabled
N/A
11
Start Flash Policy Server
Server Settings > WebSocket
Enabled
Disabled
N/A
12
Allowed SQL (all settings)
Data & Services > Data Sources ><database> Advanced Settings
Enabled
Create, Drop, Alter, Grant,Revoke, Stored Procedures are disabled
Retained if specified
13Enable Robust Exception Information
Debugging & Logging > Debug Output Settings
Disabled 
Disabled
Overwritten
14Enable CFSTAT
Debugging & Logging > Debug Output Settings
Enabled
Disabled
Overwritten
15
Select the type of
Administrator authentication
Security > Administrator 
Use a single password only
Separate user name and password authentication (allows multiple users)
N/A
16
Enable RDS Service
Security > RDS
Configurable at install time 
Disabled
N/A
17
Select the type of RDS
authentication
Security > RDS
Use a single password only
Separate user name and password authentication (allows multiple users)
N/A
18
Allowed IP addresses for ColdFusion Administrator access
Security > Allowed IP Addresses
Not available at install time
Available at install time
N/A



Ray has created a very nice Administrator Extension. The details can be found as Security Profile Admin Extension for ColdFusion 10
Related Entries:
How to Secure ColdFusion Session Cookies with CF 10
New Improved CFLogin
New way to add Sandbox permissions for Users with RDS access
Improved Session Management in ColdFusion 10
ColdFusion 10 Hot-Fix Installer

Monday, April 9, 2012

Cryptography & Invalid Key Size Exception

You might find this known but I thought would add a quick note on this topic. Recently I was debugging one of the issue reported, which turned out to be related.

When using cryptography functions you might get "InvalidKeyException Illegal key size" exception. This is not an error due to wrong API or in correct configurations.

To fix this problem, you will need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files (available at Oracle). If you don't, the keysize is limited due to US export laws. Hence many algorithms like AES etc will produce "Invalid key size" exception.

Install these policy files, as follows:

1. Download the policy files for the JDK being used.
2. Unzip the file.
3. Copy these files to the <JDK_HOME>jre/lib/security directory.
4. Start server.


This will fix your server. Read further to know how to check for these policy files.

How To Check For Unlimited Strength or Unrestricted Jurisdiction Policy Files

When you install or update JDK, the default jurisdiction policy files are installed. These files allow limited cryptography. Your applications may require you to install unrestricted versions of these files (local_policy.jar and US_export_policy.jar).
To check if you are using unrestricted version or not, copy these files from /jre/lib/security to a temporary location and use these commands

jar -xvf /local_policy.jar
jar -xvf /US_export_policy.jar


Read default_local.policy and default_US_export.policy to find out the jurisdiction policy that is being used. If you see grant { permission javax.crypto.CryptoAllPermission; }; entry in the file then you are using unrestricted version.

Thursday, April 5, 2012

Cookies Max-Age vs Expires & newer Jboss


Have you noticed some of the cookie headers sent on jboss 6 or 7. If you do, you will see that expires attribute is missing and a new attribute Max-Age is added. 
I will explain what these mean and why should you care- 
  • Expires sets an expiry date for when a cookie gets deleted. This is a time from GMT.
  • Max-age sets the time in seconds for when a cookie will be deleted
Expires was being used to specify cookie age. However as in Http 1.1 expires was deprecated, Max-Age is the replacement for this. Now instead of specifying a date, you can just specify the number of seconds a cookie should live. By setting any of these two you can persist cookies. If nothing is set then the cookie lives till browser close. i.e. it behaves like session cookie. 

Example using expires: 

set-cookie = "foo=bar;path=/;expires=Fri, 06-Apr-2012 13:30:00 GMT";
same using Max-Age

set-cookie = "foo=bar;path=/;max-age=60";

Now tricky part is Max-Age is not supported in most versions of Internet Explorer (IE). IE6, IE7, IE8 don't support it, where as rest all browsers support both. These browsers that support Max-Age will ignore expires. 

Solution For JBoss :
To instruct JBoss for always setting expires attribute you should add the following jvm property 

-Dorg.apache.tomcat.util.http.ServerCookie.ALWAYS_ADD_EXPIRES=true
There are lot more configurations which can be done for cookie properties. For a complete list of those see Apache configuration documentation.

Tuesday, April 3, 2012

ColdFusion 10 & Hot-Fix Installer



How many times you have wished there was a simpler way to apply hot-fixes? How many times number of steps involved have caused inconvenience and errors in deployment? The debugging required in most of these cases really takes a long time. Same is true for uninstalling a particular hot-fix. 


With ColdFusion 10, all of this will be solved and hot-fix installation will becomes a one click activity. Hot-fix installer (updater) is bundled within ColdFusion 10. This will eliminate the hassles of copying files, deleting files, different hot-fixes like cumulative, security etc. There will be always a single cumulative hoti-fix available. Hot-fix installer will take care of all the work required to update the server and will also keep the back up.
There is a new section in Administrator console Server Update -> Updates. Following features are added in hot-fix installer – (See Figure 1,2) 

  •  You can check from updates using “Check for updates” button,
  • You can see the update level and available hot-fixes from within administrator it self,
  • You can do manual install or automatic install of hot-fix,
  • You can configure it to check for updates automatically,When this is selected, on Administrator console, right top corner an icon mentions how many updates are available. (See Figure 3)
  • You can schedule email alerts to be raised whenever there is a new hot-fix available,
  • You can also set up an internal hot-fix update location from within intranet.

Figure1, Hot-Fix Installer







Figure 2, Hot-Fix Installer Settings





Figure 3, Hot-Fix Installer, Check for updates automatically




ColdFusion 10 Beta is available, try it now and see how easy it is to patch the server and keep them up to date.


Related Entries:
How to Secure ColdFusion Session Cookies with CF 10
New Improved CFLogin
New way to add Sandbox permissions for Users with RDS access
Improved Session Management in ColdFusion 10
ColdFusion 10 Secure Profile