Something has Changed: Profile Stat Updating

Question

Something has changed with the timing of the daily updates of Profile Stat data.  Up until a few weeks ago, my previous days caching activity was updated in Project-GC by 7:00 am MST.  As each day progressed the time of day is getting later and later.  It's now past 2:00 pm MST when I can see updates to my previous days caching activity.

As a paying member (and contributor) to Project-GC for many years, I'm very frustrated and disappointed with this degradation in service.

Please advise why caching stats cannot be updated in a more timely and consistent basis on a daily basis?

Answer 1

There has never (by design) been a fixed update time of anyone's statistics. When over 24 hours old, it's queued up for a refresh. When that refresh is done, the stats get a new timestamp and when that's 24 hours old the process repeats again.

Comments

So how is it that for well over six months, my daily stats were updated each and every day by 7:00 am MST and within the past few months, the daily update when from 7:00 am to 8:30 am to 11:30 am.  This past week it's been well past 2:00 pm MST and yesterday it was closer to 5:00 pm MST.   In 2016, I made a significant cash contribution to Project-GC.  I've not yet made such a contribution in 2017.  Could this be the issue?  Do members that contribute to Project-GC receive priority in the data update queue?

---

Paying members get daily updates and non-paying members only weekly updates, but not apart from that. None of the updates happen on a fixed time. It is likely that the update time will creep forward, since a new update is scheduled when the last one is 24 hours old, and the processing time of the update will add some time.

---

I usually see mine same day but about 21:00 - 23:00.

Answer 2

Not being involved in the support of this site I would like to add the following for a better understanding of the process (at least I hope).
In my view running scheduled processes like the regular refresh of the statistics can be managed in 2 ways.
1. Give each batch (refresh for an individual user) a fixed starting time to run in a given frequency (day or week) and run the refresh at that time.
2. Regularly check whether the latest refresh is older than the given frequency. If this is the case, run the refresh.

For both cases: Because we are dealing with a system using batches (so not realtime) it is more precise to say: "add the refresh to the queue of batches/processes to be run" instead of "run the refresh".

My understanding is that RPStew expects option 1, where pinkunicorn explains that the system was designed for option 2.

As mentioned by pinkunicorn, option 2 implicates that the refresh time moves a bit ahead in each refresh, because
- the refresh itself takes time.
- refresh start will be later than the moment of adding to the queue of processes
- the new refresh date/time will be the finishing time of the refresh.
- (not mentioned, but maybe also impacting the last refresh time) downtime of the system can cause a delay in starting the actual refresh.

My assumption is that option 1 has some disadvantages over option 2:
- Scheduling of the refreshes will be more complex, because of the assignment of the fixed starting time which should (for performance reasons) be nicely spread over the day.
- In case of downtime of the system (planned or not) the queue of processes to run will grow fast (including all refreshes that already passed the fixed start time). This could cause delays also for fixed starting times outside the downtime because of priority for earlier fixed starting times. I assume it is even possible to have an overflow of the queue for processes to be run.
These disadvantages could be mitigated, but that will cost some or even a lot of money.

BTW, I noticed before there is already some efficiency implemented for not very active users. Users without any activity or without anybody having interest in their statistics have a lower refresh rate (than once per week).
As a proof I just checked a not very active user: statistics were last created over 6 weeks ago. I checked again about 30 minutes later: stats are refreshed today.

Answer 3

So many comments here about delays of update.

Doing things in batch is also good for planet. Don't think the immediate status is at no cost. For most of us who leave in harmony with nature and slow things, why would you have real time status..?

On top of that project-GC is not only using batches, so why not blaming also geocaching.com. Don't you pay geocaching.com too and/or have also paid for GSAK(or any other in the past with you at the command for transmitting, refreshing, statisticking ..? :-)

By the way project-GC is advising regularly in a banner when the data may have been affected by delays

On my side i'm always happy for the service rendered versus price with a generally up-to-date stat after noon

Comments

Pepageo, please re-read my original question.  I've never requested real time updates from Project-GC.  I'm fine with a batch process.  What I'm asking for is consistency to the timing of the batch process.  A batch process normally runs on a predetermined schedule.  The issue I'm having with Project-GC is that in the recent months, their batch process has become extremely inconsistent and the timing varies from day to day.  As I mentioned before, this was never the case with Project-GC.  I could always count on the batch process being complete by a certain time the next day.  This is now not the case...thus the reason for my statement that "something has changed".

---

Is it really important to have your daily update without a little delay ?

---

DaneteYaourth, as a paying member for a few years and as a contributor who has donated cash to assist with keeping the site going, I have three simple expectations of Project-GC:

1) That the data is accurate
2) That access to the web site is reliable
3) That updates to User Profile data is done on a consistent timeframe each day.

My most recent experience saw my stats getting updated around 8:00 pm MST on Saturday night April 29th with the next update not taking place until around 12:30 pm MST on Monday afternoon, May 1st.  That's a 30 hour gap...which in my opinion is unacceptable.  In my opinion, 30 hours is not a little delay especially with the computing power that exists with current server technology.

---

totally agree with that.
I had my refreshes always daily at 12.00 EST and now it's gone somwhere into the night. I have the same feeling, that the 24 hours refresh is not working well, so I'm disappointed with that issue, too.
Something has changed. That's fact. And it's also fact, that some People are not very happy with that, so could you please catch up our wishes? @pinkunicorn