AT Server requires restart every morning
Every morning we come in to start the day and we have to restart the AT service to get connected. Connecting directly to MySQL is fine which suggests that it is the AT Server which is failing.
Any ideas? Below is a sample of the log file (which is filled with the same error, just differnt key each time)
org.apache.lucene.store.jdbc.JdbcStoreException: Failed to execute sql [insert into lucene_index (name_, value_, size_, lf_, deleted_) values ( ?, ?, ?, current_timestamp, ? )]; nested exception is com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '_1036.fnm' for key 1
INFO | jvm 1 | 2008/09/03 19:35:08 | com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '_1036.fnm' for key 1
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2870)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1573)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1404)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1318)
INFO | jvm 1 | 2008/09/03 19:35:08 | at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1303)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.store.jdbc.support.JdbcTemplate.executeUpdate(JdbcTemplate.java:159)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.store.jdbc.index.AbstractJdbcIndexOutput.close(AbstractJdbcIndexOutput.java:47)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.FieldInfos.write(FieldInfos.java:259)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:176)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:88)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:709)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java:656)
INFO | jvm 1 | 2008/09/03 19:35:08 | at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:402)
INFO | jvm 1 | 2008/09/03 19:35:08 | at agiletrack.server.persistence.GenericFullTextIndexer.closeWriter(GenericFullTextIndexer.java:78)
Mark
—Posted by Software on Sep 4, 2008
|
|
Well, the error you're seeing with the Lucene index can probably be solved by deleting the lucene_index table. It seems that over time the data in the table becomes somehow corrupt. I should probably try getting an updated version of Lucene and the database driver being used to see if that help. Anyway, if the table is deleted, it will be recreated the next time the server starts.
I can't say whether or not that is why you're needing to restart the server every morning. Are there more symptoms to the problem—e.g. the server refuses connection in the morning, so client can log in?
If possible, it would be helpful to get a thread dump from the server process at the time it's having trouble. If you can find the process ID (it would be a java process), you can run something like "jstack $PID > threads.log" to get a log of all the threads. Then, if you can email it to support (at) agiletrack (dot) net, I'll take a look at it.
Thanks.
—Posted by Adam Lane on Sep 8, 2008 at 7:26:21 AM