[SOLVED]Unable to check update due to a system error: time data 'Mo 20 Jun 2016' does not match format '%a %d %b %Y'

Hello,

i become this error message, if i click on Software Update.

        Traceback (most recent call last):

File “/opt/rockstor/src/rockstor/storageadmin/views/command.py”, line 177, in post
return Response(update_check(subscription=subo))
File “/opt/rockstor/src/rockstor/system/pkg_mgmt.py”, line 154, in update_check
version, date = rpm_build_info(pkg)
File “/opt/rockstor/src/rockstor/system/pkg_mgmt.py”, line 99, in rpm_build_info
bdate = datetime.strptime(dstr, ‘%a %d %b %Y’)
File “/usr/lib64/python2.7/_strptime.py”, line 325, in _strptime
(data_string, format))
ValueError: time data ‘Mo 20 Jun 2016’ does not match format ‘%a %d %b %Y’

I'm not familar with Phyton or other programming languages.

And there are more error messages like this. I can’t add members, disks, pools or shares.

Hmm, this is a localisation issue. It seems that we need to import the user’s system locale settings somewhere before handling strings like these. @suman/@Flyer/@phillxnet, what would be the best place to put this?

Hi @Uwe_Ganswig,
can you please perform this rpm -qi rockstor on your Rockstor machine shell and paste here its output???
Thanks @sfranzen

P.s: that “Mo 20 Jun 2016” is really strange, seems like a normal US date with just a missing “n” on Monday

It’s a coincidence, Uwe appears to have a German locale where Monday is Montag.

1 Like

Yes, it is a German locale.

And this is what rpm write:

Name        : rockstor
Version     : 3.8
Release     : 14
Architecture: x86_64
Install Date: Do 13 Okt 2016 17:29:45 CEST
Group       : RockStor
Size        : 84379447
License     : GPL
Signature   : RSA/SHA1, Mo 20 Jun 2016 02:56:52 CEST, Key ID 032b764d08795b09
Source RPM  : rockstor-3.8-14.src.rpm
Build Date  : Mo 20 Jun 2016 02:46:10 CEST
Build Host  : be1482955b6b
Relocations : /opt
Packager    : RockStor, Inc. <support@rockstor.com>
Vendor      : RockStor, Inc.
Summary     : RockStor -- Store Smartly
Description :
RockStor -- Store Smartly

If you can see, I’ve installed Rockstor again.

At first GUI-Start this error messagebox comes after click on submit:

      Traceback (most recent call last):

File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 40, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 245, in post
return self._update_disk_state()
File “/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/transaction.py”, line 371, in inner
return func(*args, **kwargs)
File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 62, in _update_disk_state
disks = scan_disks(settings.MIN_DISK_SIZE)
File “/opt/rockstor/src/rockstor/fs/btrfs.py”, line 1052, in scan_disks
dmap[‘SIZE’] = int(float(size_str[:-1]) * 1024 * 1024)
ValueError: invalid literal for float(): 225,5

Ok,
let’s call @phillxnet for “Disksland”, while opening an issue for locale to US / patch

1 Like

Hi @Uwe_Ganswig, asking one last test:
can you provide output from shell locale cmd ? thanks

Why am I asking this?? Had some testing on a python shell on Rockstor dev machine with this code and all was fine, returning expected 2016-Jun-21 (repeated same experiment with locale it_IT.utf8 and right italian date format and it was fine too)

    from datetime import (datetime, timedelta)
    import locale
    locale.setlocale(locale.LC_ALL, 'de_DE.utf8')
    print(locale.getlocale())
    l = 'Build Date  : Mo 20 Jun 2016 02:46:10 CEST'
    dfields = l.strip().split()
    dstr = ' '.join(dfields[3:7])
    print(dstr)
    bdate = datetime.strptime(dstr, '%a %d %b %Y')
    print(bdate)
    bdate += timedelta(days=1)
    print(bdate)
    date = bdate.strftime('%Y-%b-%d')
    print(date)

M.

Here are the output from locale

[root@localhost ~]# locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

One Question more. Isn’t the WebUI translated?

TIA,
Uwe

Hi @Uwe_Ganswig & @sfranzen too, finally got it :grin:
There’s a bug in python not reading current locale.
Having not all Rockstor users german locale we can’t setlocale to de_DE, so here the hack:

from datetime import (datetime, timedelta)
import locale
print(locale.getlocale()) #gets none,none
locale.setlocale(locale.LC_ALL,'') #fix missing locale
print(locale.getlocale()) #here we go :D
l = 'Build Date  : Mo 20 Jun 2016 02:46:10 CEST'
print(locale.getlocale())
dfields = l.strip().split()
dstr = ' '.join(dfields[3:7])
print(dstr)
bdate = datetime.strptime(dstr, '%a %d %b %Y')
print(bdate)
bdate += timedelta(days=1)
print(bdate)
date = bdate.strftime('%Y-%b-%d')
print(date)

Going to PR asap :wink: /EDIT : @suman we require having one locale.setlocale(locale.LC_ALL,'') to force python read real locale and have it in every part of Rockstor, where to put this two lines patch?

P.S.: german locale

[root@rockstone build]# ls a
ls: Zugriff auf a nicht möglich: Datei oder Verzeichnis nicht gefunden

:fearful:

2 Likes

As I’ve seen the locale is imported in the _strptime.py

"""
import time
**import locale**
import calendar
from re import compile as re_compile
from re import IGNORECASE
from re import escape as re_escape
from datetime import date as datetime_date
try:
    from thread import allocate_lock as _thread_allocate_lock
except:
    from dummy_thread import allocate_lock as _thread_allocate_lock

@Uwe_Ganswig and @Flyer this looks like a device on the system is returning an unexpected string for it’s size value!!

And we are not elegantly dealing with it.

@Uwe_Ganswig if your could paste the output of the following command executed on your Rockstor install into this forum thread it should help with working out what’s going wrong. If you could also proceed and follow the pasted output with a triple single inverted comma, each set on a line of it’s own, ie ``` it will format better within the post and aid in reading, such as I have done with the command in this post:

/usr/bin/lsblk -P -o NAME,MODEL,SERIAL,SIZE,TRAN,VENDOR,HCTL,TYPE,FSTYPE,LABEL,UUID

Cheers, quite a curiosity this one.

1 Like

@phillxnet that is always related to locale, check this:
First wit en_US locale

NAME="vda3" MODEL="" SERIAL="" SIZE="17.5G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="btrfs" LABEL="rockstor_rockstor" UUID="b4394e03-d4ee-4be3-92ae-47b3283b7ec4"
NAME="vdb" MODEL="" SERIAL="" SIZE="12G" TRAN="" VENDOR="0x1af4" HCTL="" TYPE="disk" FSTYPE="btrfs" LABEL="Data" UUID="54191fe3-1179-4766-8f18-d9952157ef46"
NAME="vdc" MODEL="" SERIAL="" SIZE="12G" TRAN="" VENDOR="0x1af4" HCTL="" TYPE="disk" FSTYPE="btrfs" LABEL="Data" UUID="54191fe3-1179-4766-8f18-d9952157ef46"

then with de_DE locale, pls note

NAME="vda3" MODEL="" SERIAL="" SIZE="17,5G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="btrfs" LABEL="rockstor_rockstor" UUID="b4394e03-d4ee-4be3-92ae-47b3283b7ec4"
NAME="vdb" MODEL="" SERIAL="" SIZE="12G" TRAN="" VENDOR="0x1af4" HCTL="" TYPE="disk" FSTYPE="btrfs" LABEL="Data" UUID="54191fe3-1179-4766-8f18-d9952157ef46"
NAME="vdc" MODEL="" SERIAL="" SIZE="12G" TRAN="" VENDOR="0x1af4" HCTL="" TYPE="disk" FSTYPE="btrfs" LABEL="Data" UUID="54191fe3-1179-4766-8f18-d9952157ef46"

On en_US size get point separator, with de_DE comma separator. Because of that getlocale() python error your system is on de_DE, but python checks values/data thinking like en_US.

Same if you have locale to it_IT or fr_FR es_ES etc etc, but not your en_GB locale because US & GB have both decimal with point (looking to this nice wikipedia map I suppose probably related to Anglo-Saxon countries / not influeced by UK culture- not anglo-saxon countries…closing history channel eheh)
Wikipedia decimal mark page

1 Like

Hi all @Uwe_Ganswig, @sfranzen @phillxnet and @suman,
before being scolded by Guido van Rossum :scream: let’s amend my last sentence about locale bug in Python:

  • Thinking about possible versions related issues had a fresh new latest 2.7.12 python version (not on Rockstor machine)
  • Had same tests
  • Having same results finally said “Ok, better reading line by line Python locale docu” and…shame on me, that locale.getlocale() (same func is used over _strptime.py get_lang to handle localization)returning ('None', 'None') is well documented and desired behavior

Reading locale reference (https://docs.python.org/2/library/locale.html#background-details-hints-tips-and-caveats) Python doesn’t assume real machine locale, but the C/POSIX locale (decimal char and times same to en_US), so having it working with real machine locale requires a locale.setlocale(locale.LC_ALL,'')

Finally, Python is ok and we’re back to my prior question : @suman where to have this so that it can spread all around Rockstor? :slight_smile:

Mirko

1 Like

OK, here are the informations:




NAME="sda" MODEL="WDC WD2500BEVT-0" SERIAL="WD-WXN1E61ACHZT" SIZE="232,9G" TRAN="sata" VENDOR="ATA" HCTL="0:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sda1" MODEL="" SERIAL="" SIZE="200M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="vfat" LABEL="" UUID="EBDA-BDC7"
NAME="sda2" MODEL="" SERIAL="" SIZE="500M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="ext4" LABEL="" UUID="4e4dedd2-79a6-4e97-8948-593622b23271"
NAME="sda3" MODEL="" SERIAL="" SIZE="6,8G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="swap" LABEL="" UUID="5ec0db0e-56c2-4167-a972-3c2d352b70a6"
NAME="sda4" MODEL="" SERIAL="" SIZE="225,5G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="btrfs" LABEL="rockstor" UUID="76b767a0-8b12-4a8e-86c3-d95e174b0c05"
NAME="sdb" MODEL="ST3750528AS    " SERIAL="6VP0HK89" SIZE="698,7G" TRAN="sata" VENDOR="ATA     " HCTL="4:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sdb1" MODEL="" SERIAL="" SIZE="500M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="ext4" LABEL="" UUID="7c40e30b-4aaf-4ac4-ad30-638e833b6109"
NAME="sdb2" MODEL="" SERIAL="" SIZE="698,2G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="LVM2_member" LABEL="" UUID="Gxz12z-XYSI-Ppy2-MK8W-Pxd1-cWra-fhaW16"
NAME="rockstor_gphomeserver2-root" MODEL="" SERIAL="" SIZE="50G" TRAN="" VENDOR="" HCTL="" TYPE="lvm" FSTYPE="xfs" LABEL=""  UUID="4a11a9ba-ea01-4bb0-94cd-240fc79e7efc"
NAME="rockstor_gphomeserver2-swap" MODEL="" SERIAL="" SIZE="6,8G" TRAN="" VENDOR="" HCTL="" TYPE="lvm" FSTYPE="swap" LABEL="" UUID="2879563d-7b18-4867-9bcc-5b6c59d45564"
NAME="rockstor_gphomeserver2-home" MODEL="" SERIAL="" SIZE="1,3T" TRAN="" VENDOR="" HCTL="" TYPE="lvm" FSTYPE="xfs" LABEL="" UUID="7bfe9c44-1e00-4c59-88b5-0f9d9496c0c2"
NAME="sdc" MODEL="ST3750528AS     " SERIAL="6VP096SJ" SIZE="698,7G" TRAN="sata" VENDOR="ATA     " HCTL="5:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sdc1" MODEL="" SERIAL="" SIZE="698,7G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="LVM2_member" LABEL="" UUID="Ab192Y-MJDr-ROhl-WlmC-b1OF-OH5J-3krLl9"
NAME="rockstor_gphomeserver2-home" MODEL="" SERIAL="" SIZE="1,3T" TRAN="" VENDOR="" HCTL="" TYPE="lvm" FSTYPE="xfs" LABEL=""  UUID="7bfe9c44-1e00-4c59-88b5-0f9d9496c0c2"

This comes out if I let the installer cofiguring the partitions itself. This produced an error at first WebUI-Screen!

I can send the Rockstor.log if it helps.

Uwe

@Uwe_Ganswig Thanks for the reply and for reporting the issue; this output confirms the findings of @Flyer re “,” rather than “.” in the “SIZE=“225,5G”” string interpretation as a result of locale setting and default Python function. It looks like @Flyer is on top of this one so I’ll back out.

No unfortunately not, but please see the following forum thread for future plans and a discussion on this:

@Flyer Thanks for the schooling, nicely demonstrated. I’m afraid I just dove in re your “…let’s call @phillxnet …” prompt without reading the thread carefully enough; nice find this one.

That’s what I was thinking about as well, but I don’t know whether there is one “right” way. Perhaps the cleanest way is to create the ultra short module

from locale import setlocale, LC_ALL

setlocale(LC_ALL, '')

and use this wherever we handle localised stuff? Django should already handle this appropriately, so the trouble is hopefully mostly within the Python utility modules (system folder).

Hi @Uwe_Ganswig, can you please post you /etc/locale.conf content?

thanks

This is the content;

LANG="de_DE.UTF-8"

Thanks,
I asked this because I’m trying to reproduce your same issue (with it_IT.UTF-8 and not DE, just to understand what I read eheh)

M.