Error notifications and status notifications - email

Currently I’m in the command line of my rockstor device on nearly a daily basis as I work through some ideas, so I’m able to see that ‘I’ve got mail’ about the fact that a hdd has some pending sector counts.

Ideally I’d have rockstor configured to send me an email when things like this happen. It’s good to know asap that you have a potentially sick hdd and the kernel is already smart enough to send mail to root, let’s get that mail to my gmail address :wink:

Other important information might be notification of reaching a configurable capacity threshold. Errors found during scrubs. Errors during replication jobs. I’m sure there’s more.

The last thought is a scheduled status notification. I came from ZFS on napp-it where I would receive a weekly email telling me the status of my pools (basically zpool status -v which also happens to have the status of the latest scrub as well as checksum error information) and the usage of my pools.

In other solutions just the ability to give the hostname/ip of the relaying smtp server, the destination email address, and the from email address is what’s provided and that’s always been fine for me. Perhaps you could consider an smtp relay rockon to fill out the solution to give us the option of using our own relay or one that’s built in.

@seijirou Yes this is a great Idea. I won’t create an Issue right now as not sure which way to go on it. The Rock-on idea is interesting and mail servers can be quite tricky security wise. If a Rock-on was created maybe it could be configured by default to serve this process initially, ie pre-configured as a satellite and setup to serve only the Rockstor system or any of it’s other Rock-ons.

Whichever way we go I suggest postfix if we are to use a full on server.

@suman need your input here prior to creating an issue as large design concerns and I believe we already have an outstanding email related pull request in #644 that arose from #issue633.

I have successfully configured postfix to use smtp.gmail.com as a smarthost (relayhost) with root forwarding and header and envelope rewrite. But that config really needs looking into security wise.

In case it helps my attempt is in the form of an interactive bash script for Ubuntu server:- from by github

But I am not keen on using my gmail in this context and have since moved away from that method. I heard recently of some free email services that would probably be a better idea to pre-configure to. That is free until you exceed some rather large monthly total sends. Can’t remember where I saw what seemed like a reputable reference though.

Anyone have experience with any email service providers for this purpose?

@phillxnet I use Postfix on Ubuntu to relay to gmail (smtp.gmail.com) and took some basic guidance found at http://security-24-7.com/hardening-guide-for-postfix-2-x/ for “hardening” the postfix configuration.
I’m definitely not a SME at the ground level of postifx hardening but the heavy hitters IMO are not running as root and configuring proper trusted networks (i.e. if you’re not on my LAN I’m not serving you). Keeping ‘badguys’ off my LAN is a job handled elsewhere so I assume if it’s sourcing from my LAN it’s ok so that’s how I’ve got it configured.

Relaying the email through Gmail isn’t an absolute necessity but I find it’s just easier and cleaner in the long run. It does require changing Gmail settings to use legacy authentication protocols that are considered less secure, but my recommendation to others is to create a new gmail account for the purpose of being a receiver of your email relay. From within this auxiliary account you can use gmail filters to auto-forward email out as you see fit and because it’s coming from gmail you won’t have any problems of any other email providers blocking it.

@seijirou Yes the lan only setting is a must, either that or localhost only if that serves. I wouldn’t recommned lowering your gmail authentication and I didn’t have to do this with my config but that may be unrelated. I know it took me ages to get it working as gmail is by default rather picky. You could check what settings I used and see if they help you revert that one.

Another gmail account; well that has to be good advise but I am swimming in google accounts as it is :smile:.

There is an open pull request for this right now. It won’t be merged in the next release but most likely in the one after that(3.8-2)

Thanks for sharing really good ideas here. I’ve created an issue for this as a starting point.

1 Like

+1 for that, emails are very useful

i dont know if this currently happening, but if not you could add sec to rockstor with a custom regex to filter btrfs related errors
here someone did that already, havent tested it so far, the regex might need some tweaks for newer syslog messages since 2014.

2 Likes

Hi everyone, I found a solution that I explained here:

2 Likes

@maxhq Thanks, this is a great find / contribution, and given that once email notifications are enabled all email to root is also forwarded to the configured email one can also get smartmontools warnings by adding config entries (via the GUI) such as those those indicated in this post.
example email notifications I’ve received include:-

Device: /dev/sde [SAT], 2 Currently unreadable (pending) sectors
Device: /dev/sdb [SAT], 940 Currently unreadable (pending) sectors
Device: /dev/sdb [SAT], FAILED SMART self-check. BACK UP DATA NOW!
Device: /dev/sdb [SAT], 4 Offline uncorrectable sectors

The above are from a variety of fairly sickly drives I keep around to test such things.:slightly_smiling:

We also have NUT events email notifications

ups@nutserver tripped no-comms timer or event filter
ups@nutserver tripped comms-ok timer or event filter
ups@netserver tripped comms-bad timer or event filter
ups@localhost tripped on-batt timer or event filter
ups@localhost tripped on-mains timer or event filter

It would be great to integrate your suggestion / config of logcheck and ease the config of smartmontools in future Rockstor versions, I’ll leave it to @suman to formalize that more in the issues as I know there are plans to establish a more robust GUI notification system that would presumably tie directly into the email one we already have. That way we could channel everything though that system and have better control.

Could you give us an example of the emails you have received from your logcheck install and config? Thanks. Notifications is definitely an area that can do with improvement.

2 Likes

Hello,
Why is sending mails always so difficult under Linux?
Why can’t we let Rockstor act like any other mail CLIENT?

Greetings,
Hensrik

@phillxnet Glad you like it! You are right, also various other messages are now notified, even BTRFS problems.

Example mails:

http://pastebin.com/CJedse3A
http://pastebin.com/KHzj3wHf
http://pastebin.com/utjDre5B
http://pastebin.com/q4fCWGMD

For Smartdata I found this config which i do hope to add in smartd conf file:

DEVICESCAN -a -o on -S on -n standby,q -s (S/…/…/./02|L/…/…/6/03) -W 4,35,40 -m “my@email”

I am not receiving any emails as the config suggests. Could anyone check what I am missing?

Hey @bug11. Did you setup e-mail notifications? Here’s the howto.