I restarted our openfire server trying to fix an issue with disappearing contacts. Had never done it before, I read in some internal documentation to kill java first by sudo /usr/bin/killall java. Turns out this was a different process for doing a restart of the openfire -service- when the admin console is hung up. What I did was kill java and restart the virt server, it came right back up after restart. Log rotation occurred about 5 hours later.
Now we are unable to view chat history / archived conversations from the openfire admin console (Server > Archiving > Search Archive) and our original problem persists.
System info:
Server Properties
Version: Openfire 3.9.3
Server Directory: /opt/openfire
Linux 2.6.32-220.23.1.el6.centos.plus.x86_64 #1 SMP Tue Jun 19 04:14:37 BST 2012 x86_64 x86_64 x86_64 GNU/Linux
MySQL:
mysql Ver 15.1 Distrib 5.5.36-MariaDB, for Linux (x86_64) using readline 5.1
Environment
Java Version: 1.7.0_55 Oracle Corporation -- Java HotSpot(TM) Server VM
Appserver: jetty/7.x.y-SNAPSHOT
OS / Hardware: Linux / i386
Java Memory: 273.92 MB of 851.50 MB (32.2%) used
Installed plugins:
Broadcast 1.9.0
Client Control 1.2.0
Hazelcast Clustering Plugin 1.2.0
Monitoring Service 1.4.2
Registration 1.6.0
Search 1.6.0
User Import Export 2.4.0
User Service 2.0.2
Symptoms:
- Not showing any archive search results after I restarted the server. (Last chat is the one right before I restarted)
- Results do not display any data on the conversation: Participants, message count, date and duration are null / incorrect, no chat messages displayed
** Example:
Participants: null,
Message Count: 0
Date:
Duration: < 1 minute
- Under Archive Settings, the indexing info is not updating. Always shows same count:
** Example:
Current Search Index - The current size of the message index. 153.5 MB
Archived Message Count - The total number of archived messages. 5029007
Archived Conversation Count - The total number of archived conversations. 610704
- View PDF link on the right actually still shows the chat logs for ones I try and bring up, but no new conversations since I restarted the server.
- Error.log shows conversation not found errors: (I'll post error log in another note)
** Example:
org.jivesoftware.openfire.archive.ConversationUtils - Conversation not found: 610513
org.jivesoftware.util.NotFoundException: Conversation not found: 610513
What I've tried already:
- Verified it is still logging; /opt/openfire/logs shows file jive.audit-20150509-000.log from today
- Rebuild index. I did get an error a couple times ""TypeError: element.getInlineOpacity is not a function", but then after restarting monitoring service plugin and trying again, it showed is started rebuilding successfully.
- Reload monitoring service plugin
- (Tried the above two a few times and in different order, no fix.)
- In this post, message archiving broken in 3.8.0 , I saw something about editing conversation IDs to fix archiving / indexing. Copied ofID.ibd to local PC, opened it up and it's full of NULLs. Not sure if it's corrupt or just empty.
Not sure where to go next. I have no knowledge of MySQL or any DB stuff like that, also I am a linux noob. Can someone help point me in the right direction? I do have backups but don't know what I would want to restore at all.
Thanks in advance!