CPU usage looks inconsistent?

McFaul · February 12, 2024, 8:35pm

the top half seems to show it averaging 50%… but in the bottom half none of the cores seem to be doing anything much at all?

Its an intel i5-13500 CPU

Hooverdan · February 12, 2024, 11:10pm

From what I can tell, the widget seems to be using only the setup in the data collector here:

github.com

rockstor/rockstor-core/blob/4d60a2c683140aaa10a8e5b201157163cc9a898b/src/rockstor/smart_manager/data_collector.py#L683-L703


      
          def send_cpu_data(self):
          
          
    while self.send_cpu:
                  cpu_stats = {}
                  cpu_stats["results"] = []
                  vals = psutil.cpu_times_percent(percpu=True)
                  ts = datetime.utcnow().replace(tzinfo=utc).isoformat()
                  for i, val in enumerate(vals):
                      name = "cpu%d" % i
                      cpu_stats["results"].append(
                          {
                              "name": name,
                              "umode": val.user,
                              "umode_nice": val.nice,
                              "smode": val.system,
                              "idle": val.idle,
                              "ts": str(ts),
                          }
                      )
                  self.emit("cpudata", {"key": "cpuWidget:cpudata", "data": cpu_stats})
                  gevent.sleep(1)

i.e.,

psutil.cpu_times_percent(interval=None, percpu=true)

which indicates the CPU percentage by CPU, but I could not find a reference where the percpu was set False to get the average CPU percentage use. So, it seems to calculate the average CPU utilization from per CPU usage, more in the windows way (all utilization rolls up to 100%) than, I guess, the Linux way (20 cores at 100% would result in an “average” of 2000%). I think, it does that around here …

github.com

rockstor/rockstor-core/blob/4d60a2c683140aaa10a8e5b201157163cc9a898b/src/rockstor/storageadmin/static/storageadmin/js/views/dashboard/cpu_usage_widget.js#L319-L324


      
          updateAvgCpuGraph: function(data) {
              var _this = this;
              //We use old getAvgCpuUsage function then
              //delete first line chart element and push new one
              //If current time is multiple of 15 secs render it
              var avgcpu = _this.getAvgCpuUsage(data);

and in that function executes a group by

github.com

rockstor/rockstor-core/blob/4d60a2c683140aaa10a8e5b201157163cc9a898b/src/rockstor/storageadmin/static/storageadmin/js/views/dashboard/cpu_usage_widget.js#L355-L379


      
          getAvgCpuUsage: function(data) {
              var _this = this;
              var tmp = _.groupBy(data, function(d) {
                  return d.ts;
              });
              return _.map(_.keys(tmp), function(key) {
                  var ds = tmp[key];
                  if (ds.length > 0) {
                      var s = _.reduce(ds, function(sum, d) {
                          return d.idle + sum;
                      }, 0);
                      var avg = s / ds.length;
                      return {
                          idle: avg,
                          ts: key
                      };
                  } else {
                      return {
                          idle: 0,
                          ts: key

This file has been truncated. show original

I am wondering whether during that average system calc (upper graph) something goes wrong when the data from 20 cores are involved,

McFaul · February 13, 2024, 3:02am

Thanks for looking into that.

I would certainly hope so (the the average is wrong), because I’d be concerned if the CPU really were averaging 50% usage when the NAS is completely idle

nothing reading, nothing writing, no scrubs or balance, no services other than SMB enabled.

Tex1954 · February 13, 2024, 9:20am

It’s been like that for a VERY LONG time… I rather ignore it… It’s just one of those low priority things in the GUI compared to all the huge work upgrading everything else I guess.

McFaul · February 13, 2024, 9:48am

Oh, do you know if there is an open bug for it? I dont know ANYTHING about git or any of the bug tracking type stuff. but even if it is a low priority one, it would be good to have a record of it somewhere.

Hooverdan · February 13, 2024, 5:09pm

@McFaul I checked and could not find an open Issue on github for this, though I think I remember it has been brought up here on the forum a couple of times. In any case, I have opened a new issue on github for this:

github.com/rockstor/rockstor-core

CPU widget - average vs. CPU-specific utilization don't seem to match up

opened 05:06PM - 13 Feb 24 UTC

Hooverdan96

This issue has been discussed a couple of times in the past on the Rockstor foru…ms. Creating this issue based on the latest thread opened by user [McFaul](https://forum.rockstor.com/u/McFaul): [cpu usage looks inconsistent](https://forum.rockstor.com/t/cpu-usage-looks-inconsistent/9282) It appears that the average CPU usage is not calculated correctly, since at the same time the CPU-specific utilization is showing a either very minimal or non-existent. I referenced some of the coding areas for this widget (see forum thread as well), which might point towards a possible solution by using a secondary data collector for average system CPU usage vs. backwards calculating it from the CPU specific one: utilization by CPU ``` psutil.cpu_times_percent(interval=None, percpu=true) ``` vs. ``` psutil.cpu_times_percent(interval=None, percpu=False) ``` Finally, once this issue is addressed, these two requests could also be considered (via separate pull requests, but could be considered by whoever delves into this topic already): [Feature Request: Graph/Widget Changes](https://github.com/rockstor/rockstor-core/issues/783) [[New Feature] New Dashboard widget - Top running processes](https://github.com/rockstor/rockstor-core/issues/1672)

In the meantime, you could use the Netdata rockon if you want to get a better view of CPU utilization (both system average as well as CPU level) as well as many other parameters of your box.