Hello,

I've been trying to figure this problem out for some time now and I'm
totally stuck. The Exception in question is the following:
Caused by: java.lang.NullPointerException
at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:7009)
at
com.server.persistence.state.connectors.jdbc.JDBCCleanupStateConnector.getUUIDsByTable(JDBCCleanupStateConnector.java:140)
... 8 more

It occurs very infrequently (and unfortunately I have not been able to
reproduce it yet). I have also checked all occurences of getConnection().
Every one of them is wrapped (like below) by try {} finally {if (con !null) con.close();}. Connections are not stored locally and used by other
threads later as far as I can tell either.
The last time this problem occured, the thread in question was actually
stuck for 10 minutes (as far as I reconstruct) before the NPE was thrown.
During this time, the thread was seemingly stuck on
SocketInputStream.socketRead0 (see below for stack trace), even though I
set the sockettimeout when creating the db pool.

We're using MySQL Connector/J 5.1.24 (but there are no changes regarding
anything like this problem in 5.1.25 or 5.1.26 - and this problem has
persisted for some versions already) and Tomcat's JDBC Pool 7.0.42.

Best regards
Henning Rohlfs


@Override
public List<String> getUUIDsByTable( final String table) throws
TableNotFoundException {
Connection connection = null;
try {
connection = stateDbIo.getConnection();
final PreparedStatement ps = connection.prepareStatement(
"SELECT uuid FROM `" + table+ "` GROUP BY uuid" );
final ResultSet rs = ps.executeQuery();
final List<String> result = new ArrayList<String>();
while ( rs.next() ) { //this is where the exception is thrown
result.add( rs.getString( UUID_COLUMN ) );
}
return result;
} catch ( final Exception e ) {
throw new TableNotFoundException( table, e );
} finally {
if ( connection != null ) {
try {
connection.close();
} catch ( final SQLException e ) {
// ignore
}
}
}
}


I'm using the tomcat jdbc database pool (7.0.42) to get the connections.

stateDbIo.getConnection():
public Connection getConnection() throws SQLException {
if ( dataSource != null ) {
final Connection connection = dataSource.getConnection();
connection.setReadOnly( readOnly );
connection.setAutoCommit( true );
return connection;
}
return null;
}


Connection pool settings:
protected String initialSize = "0";
protected String maxActive = "8";
protected String maxIdle = "8";
protected String minIdle = "0";
protected String timeBetweenEvictionRunsMillis = "30000";
protected String minEvictableIdleTimeMillis = "60000";
protected String validationQuery = "SELECT 1";
protected String validationInterval = "30000";
protected String testOnBorrow = "true";
protected String removeAbandoned = "true";
protected String removeAbandonedTimeout = "60";
protected String logAbandoned = "true";
protected String jdbcInterceptors "org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;" +
"org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;" +
"org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer";
protected String connectionProperties "[connectTimeout=90000;socketTimeout=90000;]";

props = new Properties();
props.put( "url", url );
props.put( "driverClassName", driverClass );
props.put( "username", username );
props.put( "password", password );
props.put( "initialSize", initialSize );
props.put( "maxActive", maxActive );
props.put( "maxIdle", maxIdle );
props.put( "minIdle", minIdle );
props.put( "timeBetweenEvictionRunsMillis",
timeBetweenEvictionRunsMillis );
props.put( "minEvictableIdleTimeMillis", minEvictableIdleTimeMillis
);
props.put( "validationQuery", validationQuery );
props.put( "validationInterval", validationInterval );
props.put( "testOnBorrow", testOnBorrow );
props.put( "removeAbandoned", removeAbandoned );
props.put( "removeAbandonedTimeout", removeAbandonedTimeout ); //
seconds
props.put( "logAbandoned", logAbandoned );
props.put( "connectionProperties", connectionProperties );
props.put( "jdbcInterceptors", jdbcInterceptors );
return new DataSourceFactory().createDataSource( props );


Stack trace of stuck thread before it threw the exception:
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.
socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at
com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
at
com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
at
com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
- locked <0x000000063bc4b2a8> (a
com.mysql.jdbc.util.ReadAheadInputStream)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3049)
at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2157)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1932)
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3437)
at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:488)
at
com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3131)
at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2299)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2722)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
- locked <0x000000062f772b78> (a com.mysql.jdbc.JDBC4Connection)
at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
- locked <0x000000062f772b78> (a com.mysql.jdbc.JDBC4Connection)
at
com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
- locked <0x000000062f772b78> (a com.mysql.jdbc.JDBC4Connection)
at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.tomcat.jdbc.pool.interceptor.AbstractQueryReport$StatementProxy.invoke(AbstractQueryReport.java:235)
at com.sun.proxy.$Proxy1.executeQuery(Unknown Source)
at
com.server.persistence.state.connectors.jdbc.JDBCCleanupStateConnector.getUUIDsByTable(JDBCCleanupStateConnector.java:138)
...