Not being involved in the support of this site I would like to add the following for a better understanding of the process (at least I hope).
In my view running scheduled processes like the regular refresh of the statistics can be managed in 2 ways.
1. Give each batch (refresh for an individual user) a fixed starting time to run in a given frequency (day or week) and run the refresh at that time.
2. Regularly check whether the latest refresh is older than the given frequency. If this is the case, run the refresh.
For both cases: Because we are dealing with a system using batches (so not realtime) it is more precise to say: "add the refresh to the queue of batches/processes to be run" instead of "run the refresh".
My understanding is that RPStew expects option 1, where pinkunicorn explains that the system was designed for option 2.
As mentioned by pinkunicorn, option 2 implicates that the refresh time moves a bit ahead in each refresh, because
- the refresh itself takes time.
- refresh start will be later than the moment of adding to the queue of processes
- the new refresh date/time will be the finishing time of the refresh.
- (not mentioned, but maybe also impacting the last refresh time) downtime of the system can cause a delay in starting the actual refresh.
My assumption is that option 1 has some disadvantages over option 2:
- Scheduling of the refreshes will be more complex, because of the assignment of the fixed starting time which should (for performance reasons) be nicely spread over the day.
- In case of downtime of the system (planned or not) the queue of processes to run will grow fast (including all refreshes that already passed the fixed start time). This could cause delays also for fixed starting times outside the downtime because of priority for earlier fixed starting times. I assume it is even possible to have an overflow of the queue for processes to be run.
These disadvantages could be mitigated, but that will cost some or even a lot of money.
BTW, I noticed before there is already some efficiency implemented for not very active users. Users without any activity or without anybody having interest in their statistics have a lower refresh rate (than once per week).
As a proof I just checked a not very active user: statistics were last created over 6 weeks ago. I checked again about 30 minutes later: stats are refreshed today.