REST API and the (Sessions link inside OpenFire admin area) stops working after > 80k online users.
I've tested this prior to v4 OpenFire and it had the same issue. I currently have the latest version 4.0.3 and it also has this issue.
It is not exactly 80k users, but somewhere between 80k - 90k.
I have 4 tsung servers being used to load test this 2 node cluster. After approximately 80k users are online the issue happens. Even when all 110k users are online and the load on the CPUs goes down to zero I cannot use REST and it times out if I click on the Sessions link within OpenFire. I'll also add that even when there's 20k users or less, the Sessions link is terribly slow to respond and many times, will timeout.
As soon as I start dropping clients via tsung servers, once it gets below 80k users online mark, then REST and Sessions link start working again. I've tested without the Hazelcast plugin on a single server and I never have this issue. Sessions link responds very fast and REST work the entire time.
Update: I used a single tsung server to load test Node 1 of the 2 node cluster. I left Node 2 alone with no connections. I logged into Node 2's OpenFire web portal to see if clicking on the Sessions link made any difference and it did not. It timed out, even after all users were logged in and idle.