Help for a Noob for Installation

Yes, I suspected, it would not work. Adding the devices explicitly should work, too.
The way you’ve done it, works (btw, to check whether the json is consistent with all the commas and brackets, you can run it in a json linter, e.g. https://jsonlint.com/ - it obviously won’t check for the functional context, but it will ensure your key-value pairs are correct).
Seems also that you have to explicitly activate plugins (like SMART monitoring) in the conf file. So, that would also mean that the configuration file should be persisted using a share mapping.

Finally, you also need to then tell NetData to install smartctl inside the docker container (which you can see in the documentation, as an environment variable). So, something like this

...
...
                "opts": [
                    [
                        "-e",
                        "NETDATA_EXTRA_DEB_PACKAGES=smartmontools"
                    ],
                    [
                        "--cap-add=SYS_ADMIN",
                        ""
                    ],
...
...

In reading some more, since smartctl is already installed on the host, you can also expose the smartctl data to the docker container instead. That way, if you change drives, etc. you don’t have to reconfigure the container (scrutiny is set up that way, so you don’t have to explicitly attach devices to it). I am basing that, among others on this issue that was raised in one of the forums and found its way into a github issue:

OpenSUSEs implementation of smartctl seems to store the smart data in

/var/lib/smartmontools/

I was planning on trying this, but unfortunately on my system one of the drives decided to suddenly fail and I’m scrambling to replace it right now. Once that’s taken care of, I’ll take another look to see whether this can be done more generically.

I haven’t tested this, but here’s my view on when not running smartctl within the container, but take advantage of smartctl on Rockstor directly (so not adding the smartmontools as an environment variable):

{
    "Netdata (Test)": {
        "containers": {
            "netdata_test": {
                "image": "netdata/netdata",
                "launch_order": 1,
                "opts": [
                    [
                        "--cap-add=SYS_ADMIN",
                        ""
                    ],
                    [
                        "--cap-add=SYS_PTRACE",
                        ""
                    ],
                    [
                        "--cap-add=SYS_RAWIO",
                        ""
                    ],
                    [
                        "--security-opt",
                        "apparmor=unconfined"
                    ],
                    [
                        "--net=host",
                        ""
                    ],
                    [
                        "-v",
                        "/var/run/docker.sock:/var/run/docker.sock"
                    ],
                    [
                        "-v",
                        "/proc:/host/proc:ro"
                    ],
                    [
                        "-v",
                        "/var/log:/host/var/log:ro"
                    ],
                    [
                        "-v",
                        "/var/lib:/host/lib:ro"
                    ],
                    [
                        "-v",
                        "/etc/passwd:/host/etc/passwd:ro"
                    ],
                    [
                        "-v",
                        "/etc/group:/host/etc/group:ro"
                    ],
                    [
                        "-v",
                        "/sys:/host/sys:ro"
                    ],
                    [
                        "-v",
                        "/etc/os-release:/etc/os-release:ro"
                    ]
                ],
                "ports": {
                    "19999": {
                        "description": "Port used to access the webUI port. MUST be 19999.",
                        "host_default": 19999,
                        "label": "webUI port",
                        "ui": true
                    }
                },
                "environment": {
                    "PGID": {
                        "description": "GID of the 'docker' group. See System - Identity - Groups in Rockstor's UI to find it.",
                        "label": "PGID",
                        "index": 1,
                        "default": 472
                    }
                },
                "volumes": {
                    "/etc/netdata": {
                        "description": "Netdata configuration Share. Ensure that the share provides full access to the 'docker' group.",
                        "label": "Configuration Share"
                    }
                }
            }
        },
        "description": "Netdata is a scalable, distributed, real-time, performance and health monitoring solution for Linux, FreeBSD and MacOS.<p>Out of the box, it collects 1k to 5k metrics per server per second. It is the corresponding of running top, vmstat, iostat, iotop, sar, systemd-cgtop and a dozen more console tools in parallel. netdata is very efficient in this: the daemon needs just 1% to 3% cpu of a single core.</p><p>Based on the official docker image: <a href='https://hub.docker.com/r/netdata/netdata' target='_blank'>https://hub.docker.com/r/netdata/netdata</a>, available for amd64 and arm64 architecture.</p>",
        "icon": "https://github.com/firehol/netdata/blob/master/web/images/seo-performance-64.png",
        "more_info": "See <a href='https://learn.netdata.cloud/' target='_blank'>https://learn.netdata.cloud/</a> for more info.",
        "website": "https://www.netdata.cloud/",
        "ui": {
            "slug": ""
        },
        "version": "1.0t"
    }
}

And then follow the configuration updates described in the documentation (not sure whether also changes have to be done in the netdata.conf file to activate the plugin, have not gotten that far):

Also, for NVMe devices there is a different collector:

1 Like

Sorry to hear about your failed drives. I hope you can get that taken care of soon. I follow your instructions and see how far I can get and will report back.

2 Likes

Hello @Hooverdan
I hope you were able to sort out the issue with your failed drive and no data was lost.

I was able to run some more tests on Netdata based on your last post.
Here is what I did, tried 2 versions of Netdata_Test.json container files. First file (Netdata_test1) was the container as written by you on the last post

This unfortunately did not work, I was NOT able to see the S.M.A.R.T parameters and charts on Netdata.

The second file was where I added the smartctl inside the docker container as well as explicitly adding all my 6 drives. However, I ran into an issue installing this RockOn. It tired to install but failed and I could not understand why. The json file included the following:

{
    "Netdata (Test)": {
        "containers": {
            "netdata_test": {
                "image": "netdata/netdata",
                "launch_order": 1,
                "opts": [
                    [
                        "-e",
                        "NETDATA_EXTRA_DEB_PACKAGES=smartmontools"
                    ],
                    [
                        "--cap-add=SYS_ADMIN",
                        ""
                    ],
                    [
                        "--cap-add=SYS_PTRACE",
                        ""
                    ],
                    [
                        "--cap-add=SYS_RAWIO",
                        ""
                    ],
                    [
                        "--device= /dev/sda:/dev/sda",
                        ""
                    ],
                    [
                        "--device= /dev/sdb:/dev/sdb",
                        ""
                    ],
                    [
                        "--device= /dev/sdc:/dev/sdc",
                        ""
                    ],
                    [
                        "--device= /dev/sdd:/dev/sdd",
                        ""
                    ],
                    [
                        "--device= /dev/sde:/dev/sde",
                        ""
                    ],
                    [
                        "--device= /dev/sdf:/dev/sdf",
                        ""
                    ],
                    [
                        "--security-opt",
                        "apparmor=unconfined"
                    ],
                    [
                        "--net=host",
                        ""
                    ],
                    [
                        "-v",
                        "/var/run/docker.sock:/var/run/docker.sock"
                    ],
                    [
                        "-v",
                        "/proc:/host/proc:ro"
                    ],
                    [
                        "-v",
                        "/sys:/host/sys:ro"
                    ],
                    [
                        "-v",
                        "/etc/os-release:/etc/os-release:ro"
                    ]
                ],
                "ports": {
                    "19999": {
                        "description": "Port used to access the webUI port. MUST be 19999.",
                        "host_default": 19999,
                        "label": "webUI port",
                        "ui": true
                    }
                },
                "environment": {
                    "PGID": {
                        "description": "GID of the 'docker' group. See System - Identity - Groups in Rockstor's UI to find it.",
                        "label": "PGID",
                        "index": 1,
                        "default": 472
                    }
                }
            }
        },
        "description": "Netdata is a scalable, distributed, real-time, performance and health monitoring solution for Linux, FreeBSD and MacOS.<p>Out of the box, it collects 1k to 5k metrics per server per second. It is the corresponding of running top, vmstat, iostat, iotop, sar, systemd-cgtop and a dozen more console tools in parallel. netdata is very efficient in this: the daemon needs just 1% to 3% cpu of a single core.</p><p>Based on the official docker image: <a href='https://hub.docker.com/r/netdata/netdata' target='_blank'>https://hub.docker.com/r/netdata/netdata</a>, available for amd64 and arm64 architecture.</p>",
        "icon": "https://github.com/firehol/netdata/blob/master/web/images/seo-performance-64.png",
        "more_info": "See <a href='https://learn.netdata.cloud/' target='_blank'>https://learn.netdata.cloud/</a> for more info.",
        "website": "https://www.netdata.cloud/",
        "ui": {
            "slug": ""
        },
        "version": "1.0t"
    }
}

Also, regarding your comment below, I didnt understand which conf file needed editing. Also, I did check the dir /var/lib/smartmontools/ and it does have the data for the 6 disks in there in cvs file format and state~ file format

Let me know your thoughts.

Yes, it seems like no data was lost, and I replaced to disks over the weekend.

I am a bit short on time this week to test this myself. I have to think about the file I proposed a bit more. One thing I believe is that netdata expects smartmon data in json format and not csv (I think I read that on my since closed searches), so that approach might not work out of the box (unless additional fiddling with smartctl also then outputs the json format of that info.

when reading the documentation there are a couple of things you need to adjust in the netdata.conf file, like enabling the smartctl plugin. In order to have that persisted, you also will need to add a volume section, so you can select a share for that purpose. There you can then put the configuration file (I believe) and make changes to it.

So under your environment definition (for PGID) you want to add the volume section:

                "volumes": {
                    "/etc/netdata": {
                        "description": "Netdata configuration Share. Ensure that the share provides full access to the 'docker' group.",
                        "label": "Configuration Share"
                    }
                }

Or so I think …