metrics graphs are dropping at the end again

Sometimes the users and network graphs are dropping at the end. I assume this is another bug where we average things into the future but the future values are 0.

Did our data collection change in the past week or so in some way we can fix? I guess the alternative is to scale back how close to present we're willing to print a data point for; but that would be a shame.

added component::metrics/website priority::medium resolution::fixed status::closed type::defect labels

Trac:
Cc: N/A to karsten

You probably mean direct and bridge user graphs. The network graphs should be fine, unless there are other problems I didn't see right now.

Yes, we did change the number of days we cut off at the end of each graph. Looks like we're not cutting off enough days. Here's an example of the worst case of days we need to cut off:

• August 13: We want to graph direct or bridge usage stats for all of August 13.
• August 14, 23:00: A relay or bridge has its stats interval ending at 23:00 UTC, so the first stats interval containing the last August 13 numbers ends on August 14, 23:00 UTC.
• August 15, 17:00: Descriptors are published at least every 18 hours, so it may take until August 15, 17:00 UTC to include those stats. That's also when metrics-db grabs the extra-info descriptor and metrics-web imports it.
• August 16, 07:45: It takes until August 16, 07:45 UTC for metrics-web to finish its daily database update.
• August 16, 13:45: Graph images are cached for up to 6 hours, so it may take until 13:45 UTC until drawing a graph that correctly includes August 13.
• August 17, 00:00: When counting in days, the first graph showing user numbers from August 13 can be drawn on August 17 at 00:00 UTC.

So, to be sure, when drawing a graph, we should exclude the current day and the 3 previous days. With the latest patch we were only excluding the current day and the day before. Fixing.

In the future we may reduce the time between relays or bridges publishing a descriptor and updating the database and graphs. That won't help with stats intervals not ending at 00:00 UTC and relays publishing extra-info descriptors only every 18 hours.

Trac:
Resolution: N/A to fixed
Status: new to closed

Roger said that the network graph is affected by this problem. This could be because of the daily database update that finishes at 07:45 UTC. I should look into this some more.

Trac:
Resolution: fixed to N/A
Status: closed to reopened

Trac:

Ah, can you be more precise which network graphs you mean? In theory, network size graphs and graphs based on advertised bandwidth shouldn't be affected, but graphs based on bandwidth history might be affected. See, for example, the total relay bandwidth graph. Note how just one line drops at the end. Is that the effect you were observing?

I changed the graphing code to cut off more days from the graphs that are based on extra-info descriptors. This should fix the issue. Please re-open if it doesn't.

Trac:
Resolution: N/A to fixed
Status: reopened to closed

closed