phillxnet
(Philip Guyton)
November 14, 2020, 7:40pm
4
@GeoffA Re:
Apart from the strange times we now live in, which has inevitable affected Rockstor’s development a tad, we are awaiting a fix for our currently broken Active Directory, i.e.:
Brief description of the problem
I’m trying to join Rockstor to my AD domain, but it is erroring out.
Detailed step by step instructions to reproduce the problem
I filled out my Active Directory configuration on the services page and then clicked to start the service. I then get the below error.
Web-UI screenshot
[Capture]
Error Traceback provided on the Web-UI
Traceback (most recent call last):
File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line …
We don’t yet have an issue for this but @Flox has been considering our options re AD and it very much looks like we are going to revert our previously reverted sssd move. See the somewhat out of date docs re our prior move to sssd here:
This is a wikified post documenting the implementation of Active Directory integration in Rockstor. It was done differently before version 3.8-9 using winbind as described here . This new approach uses sssd and is a much simpler and recommended one as per RHEL/CentOS documentation .
##User story and implementation summary
User should be able to add their Rockstor system to their Active Directory domain controller. We use realmd, which provides and simple way to discover and join Active Directory…
So in short we are going to have another go at moving to sssd given it’s progress in the mean time. Post fixing this rather late but significant find I think we can more clearly offer up Rockstor 4 as having mostly feature parity with our prior CentOS offering. This should minimise the surprises for those awaiting the next stable with the assumption it can pick-up anywhere our CentOS offering left off. I’d also like to get our unit tests a little more in shape as we still have some silly failures on the 'Built on openSUSE side when we had intended, prior to the next stable release, to ensure we had the exact same unit test coverage that we had post:
rockstor:master
← phillxnet:2137_temporarily_remark_out_all_currently_failing_unit_tests
opened 10:30PM - 19 Apr 20 UTC
All currently failing unit tests on our to-be-legacy but still reference CentOS … source build are remarked out in order that we can more readily use the existing tests as indicators of disparity between our ditro bases. In almost all cases the fail/error message of the test is noted in TODO's. The vast majority failures are suspected as being caused by a lack of maintenance within the tests and several major changes such as disk/share/pool/network API changes.
Fixes #2137
See issue text for context.
Includes:
- Remark out 2 sftp sub tests, broken/out of sync with core code.
- Remark out test_btrfs_disk_import: 500 not 200.
- Fix broken test_commands.py CommandTests - core refactor bit rot. see: "[openSUSE] use zypper for package installs. Fixes #2071" (pr #2081) in 3.9.2-51. The unrelated get_pool_info() breakage was not tracked down.
- Update patch targets due to core refactoring. In pr: "Add unit testing for core network functions" #2045 at least a couple of functions were renamed. Updated these in our network API test setup.
- Remark out all network API tests as this API changed long ago.
- Remark out many whole and sub-parts of the nfs tests. Many of the failures here are likely down to disk / share API changes after these tests were written. Specifically Shares API in this case as there are many (noted in TODO: comments): "share instance ... does not exist." which suggests a failure of the mocking arrangement.
- Remark out test_share_acl as currently non working and need work.
- An attempt was made to fix the mocking required to have this test work but it was not a success. Minor minification were left in place never-the-less.
- Remark out test within test_oauth_apply - Needs updating.
- Update to account for refactoring in tested & add new needed mocks.
- Since this test was update there have been a few pool model changes. The added mocks mostly move this test back to where it was prior to the pool management code changes.
- Remove overly ambitious/broad test_raid_migration - failing.
- Fix missing space in expected error message test_raid6_crud.
- Remark out currently unworkable sub tests within test_raid6_crud.
- Our pool / disk member mocking is currently insufficient and so a range of tests now fail. Remove for to use what we can.
- Remark out currently unworkable sub tests within test_raid1_crud.
- Our pool / disk member mocking is currently insufficient and so a range of tests now fail. Remove for to use what we can.
- Remark out several sub tests within test_samba.py as need fixing.
- Remark out a sub test within test_group as need fixing or OOD.
- Remark out a sub test within test_login as non representative.
- Remark out a sub test within test_shares as incorrectly triggered.
- Remark out all but on test in test_oauth as test seem broken.
- Remark out one more test in test_snapshots as known to be duff.
- Remark out several more test in test_user as known to be duff.
# Tested on a source build via our CentOS variant via:
```
./bin/test -v 3 -p test_*
```
with the following summary:
```
Ran 203 tests in 32.252s
OK
```
and the following test output:
> test_delete_requests (storageadmin.tests.test_appliances.AppliancesTests) ... ok
> test_get (storageadmin.tests.test_appliances.AppliancesTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_appliances.AppliancesTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_appliances.AppliancesTests) ... ok
> test_auto_update_status_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_bootstrap_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_current_user_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_current_version_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_disable_auto_update_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_enable_auto_update_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_kernel_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_reboot (storageadmin.tests.test_commands.CommandTests) ... ok
> test_refresh_disk_state (storageadmin.tests.test_commands.CommandTests) ... ok
> test_refresh_pool_state (storageadmin.tests.test_commands.CommandTests) ... ok
> test_refresh_share_state (storageadmin.tests.test_commands.CommandTests) ... ok
> test_refresh_snapshot_state (storageadmin.tests.test_commands.CommandTests) ... ok
> test_shutdown (storageadmin.tests.test_commands.CommandTests) ... ok
> test_update_check_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_update_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_uptime_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_utcnow_command (storageadmin.tests.test_commands.CommandTests) ... ok
> test_get_sname (storageadmin.tests.test_config_backup.ConfigBackupTests) ... ok
> test_update_rockon_shares (storageadmin.tests.test_config_backup.ConfigBackupTests) ... ok
> test_valid_requests (storageadmin.tests.test_config_backup.ConfigBackupTests) ... ok
> test_validate_install_config (storageadmin.tests.test_config_backup.ConfigBackupTests) ... ok
> test_validate_update_config (storageadmin.tests.test_config_backup.ConfigBackupTests) ... ok
> test_get_requests (storageadmin.tests.test_dashboardconfig.DashboardConfigTests) ... ok
> test_post_requests (storageadmin.tests.test_dashboardconfig.DashboardConfigTests) ... ok
> test_put_requests (storageadmin.tests.test_dashboardconfig.DashboardConfigTests) ... ok
> test_get (storageadmin.tests.test_disk_smart.DiskSmartTests) ... ok
> test_post_reqeusts_1 (storageadmin.tests.test_disk_smart.DiskSmartTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_disk_smart.DiskSmartTests) ... ok
> test_blink_drive (storageadmin.tests.test_disks.DiskTests) ... ok
> test_btrfs_disk_import_fail (storageadmin.tests.test_disks.DiskTests) ... ok
> test_disable_smart (storageadmin.tests.test_disks.DiskTests) ... ok
> test_disk_scan (storageadmin.tests.test_disks.DiskTests) ... ok
> test_disk_wipe (storageadmin.tests.test_disks.DiskTests) ... ok
> test_enable_smart (storageadmin.tests.test_disks.DiskTests) ... ok
> test_enable_smart_when_available (storageadmin.tests.test_disks.DiskTests) ... ok
> test_invalid_command (storageadmin.tests.test_disks.DiskTests) ... ok
> test_invalid_disk_wipe (storageadmin.tests.test_disks.DiskTests) ... ok
> test_delete_requests (storageadmin.tests.test_email_client.EmailTests) ... ok
> test_get (storageadmin.tests.test_email_client.EmailTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_email_client.EmailTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_email_client.EmailTests) ... ok
> test_delete_requests (storageadmin.tests.test_group.GroupTests) ... ok
> test_get_requests (storageadmin.tests.test_group.GroupTests) ... ok
> test_post_requests (storageadmin.tests.test_group.GroupTests) ... ok
> test_post_requests (storageadmin.tests.test_login.LoginTests) ... ok
> test_adv_nfs_get (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_adv_nfs_post_requests (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_delete_requests (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_invalid_admin_host1 (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_invalid_admin_host2 (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_invalid_get (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_post_requests (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_put_requests (storageadmin.tests.test_nfs_export.NFSExportTests) ... ok
> test_get (storageadmin.tests.test_oauth_app.OauthAppTests) ... ok
> test_get (storageadmin.tests.test_pool_balance.PoolBalanceTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_pool_balance.PoolBalanceTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_pool_balance.PoolBalanceTests) ... ok
> test_get (storageadmin.tests.test_pool_scrub.PoolScrubTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_pool_scrub.PoolScrubTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_pool_scrub.PoolScrubTests) ... ok
> test_compression (storageadmin.tests.test_pools.PoolTests) ... ok
> test_delete_pool_with_share (storageadmin.tests.test_pools.PoolTests) ... ok
> test_get (storageadmin.tests.test_pools.PoolTests) ... ok
> test_invalid_requests_1 (storageadmin.tests.test_pools.PoolTests) ... ok
> test_invalid_requests_2 (storageadmin.tests.test_pools.PoolTests) ... ok
> test_invalid_root_pool_edits (storageadmin.tests.test_pools.PoolTests) ... ok
> test_mount_options (storageadmin.tests.test_pools.PoolTests) ... ok
> test_name_regex (storageadmin.tests.test_pools.PoolTests) ... ok
> test_raid0_crud (storageadmin.tests.test_pools.PoolTests) ... ok
> test_raid10_crud (storageadmin.tests.test_pools.PoolTests) ... ok
> test_raid1_crud (storageadmin.tests.test_pools.PoolTests) ... ok
> test_raid5_crud (storageadmin.tests.test_pools.PoolTests) ... ok
> test_raid6_crud (storageadmin.tests.test_pools.PoolTests) ... ok
> test_single_crud (storageadmin.tests.test_pools.PoolTests) ... ok
> test_create_samba_share (storageadmin.tests.test_samba.SambaTests) ... ok
> test_create_samba_share_existing_export (storageadmin.tests.test_samba.SambaTests) ... ok
> test_create_samba_share_incorrect_share (storageadmin.tests.test_samba.SambaTests) ... ok
> test_delete_requests_1 (storageadmin.tests.test_samba.SambaTests) ... ok
> test_delete_requests_2 (storageadmin.tests.test_samba.SambaTests) ... ok
> test_get_non_existent (storageadmin.tests.test_samba.SambaTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_samba.SambaTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_samba.SambaTests) ... ok
> test_post_requests_no_admin (storageadmin.tests.test_samba.SambaTests) ... ok
> test_put_requests_1 (storageadmin.tests.test_samba.SambaTests) ... ok
> test_put_requests_2 (storageadmin.tests.test_samba.SambaTests) ... ok
> test_validate_input (storageadmin.tests.test_samba.SambaTests) ... ok
> test_validate_input_error (storageadmin.tests.test_samba.SambaTests) ... ok
> test_delete_requests_1 (storageadmin.tests.test_sftp.SFTPTests) ... ok
> test_delete_requests_2 (storageadmin.tests.test_sftp.SFTPTests) ... ok
> test_get (storageadmin.tests.test_sftp.SFTPTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_sftp.SFTPTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_sftp.SFTPTests) ... ok
> test_clone_command (storageadmin.tests.test_share_commands.ShareCommandTests) ... ok
> test_rollback_command (storageadmin.tests.test_share_commands.ShareCommandTests) ... ok
> test_compression (storageadmin.tests.test_shares.ShareTests) ... ok
> test_create (storageadmin.tests.test_shares.ShareTests) ... ok
> test_delete2 (storageadmin.tests.test_shares.ShareTests) ... ok
> test_delete3 (storageadmin.tests.test_shares.ShareTests) ... ok
> test_delete_set1 (storageadmin.tests.test_shares.ShareTests) ... ok
> test_delete_share_with_snapshot (storageadmin.tests.test_shares.ShareTests) ... ok
> test_get (storageadmin.tests.test_shares.ShareTests) ... ok
> test_name_regex (storageadmin.tests.test_shares.ShareTests) ... ok
> test_resize (storageadmin.tests.test_shares.ShareTests) ... ok
> test_clone_command (storageadmin.tests.test_snapshot.SnapshotTests) ... ok
> test_delete_requests (storageadmin.tests.test_snapshot.SnapshotTests) ... ok
> test_get (storageadmin.tests.test_snapshot.SnapshotTests) ... ok
> test_post_requests_1 (storageadmin.tests.test_snapshot.SnapshotTests) ... ok
> test_post_requests_2 (storageadmin.tests.test_snapshot.SnapshotTests) ... ok
> test_get (storageadmin.tests.test_tls_certificate.TlscertificateTests) ... ok
> test_post_requests (storageadmin.tests.test_tls_certificate.TlscertificateTests) ... ok
> test_get (storageadmin.tests.test_update_subscription.UpdateSubscriptionTests) ... ok
> test_post_requests (storageadmin.tests.test_update_subscription.UpdateSubscriptionTests) ... ok
> test_delete_requests (storageadmin.tests.test_user.UserTests) ... ok
> test_duplicate_name2 (storageadmin.tests.test_user.UserTests) ... ok
> test_email_validation (storageadmin.tests.test_user.UserTests) ... ok
> test_get (storageadmin.tests.test_user.UserTests) ... ok
> test_invalid_UID (storageadmin.tests.test_user.UserTests) ... ok
> test_post_requests (storageadmin.tests.test_user.UserTests) ... ok
> test_pubkey_validation (storageadmin.tests.test_user.UserTests) ... ok
> test_put_requests (storageadmin.tests.test_user.UserTests) ... ok
> test_snmp_0 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_0_1 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_1 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_2 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_3 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_4 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_5 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_6 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_snmp_7 (smart_manager.tests.test_snmp.SNMPTests) ... ok
> test_delete_invalid (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_delete_valid (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_get (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_post_invalid_type (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_post_name_exists (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_post_valid (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_put_invalid (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_put_valid (smart_manager.tests.test_task_scheduler.TaskSchedulerTests) ... ok
> test_balance_status_cancel_requested (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_balance_status_finished (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_balance_status_in_progress (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_balance_status_pause_requested (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_balance_status_paused (fs.tests.test_btrfs.BTRFSTests)
> Test to see if balance_status() correctly identifies a Paused balance ... ok
> test_balance_status_unknown_parsing (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_balance_status_unknown_unmounted (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_degraded_pools_found (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_dev_stats_zero (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_device_scan_all (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_device_scan_parameter (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_dev_io_error_stats (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_pool_raid_levels_identification (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_property_all (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_property_compression (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_property_ro (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_snap_2 (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_snap_legacy (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_is_subvol_exists (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_is_subvol_nonexistent (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_parse_snap_details (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_scrub_status_cancelled (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_scrub_status_conn_reset (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_scrub_status_finished (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_scrub_status_halted (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_scrub_status_running (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_share_id (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_shares_info_legacy_system_pool_fresh (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_shares_info_legacy_system_pool_used (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_shares_info_system_pool_post_btrfs_subvol_list_path_changes (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_shares_info_system_pool_used (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_snapshot_idmap_home_rollback (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_snapshot_idmap_home_rollback_snap (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_snapshot_idmap_mid_replication (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_snapshot_idmap_no_snaps (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_snapshot_idmap_snapper_root (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_volume_usage (fs.tests.test_btrfs.BTRFSTests) ... ok
> test_get_byid_name_map (system.tests.test_osi.OSITests) ... ok
> test_get_byid_name_map_prior_command_mock (system.tests.test_osi.OSITests) ... ok
> test_get_dev_byid_name (system.tests.test_osi.OSITests) ... ok
> test_get_dev_byid_name_no_devlinks (system.tests.test_osi.OSITests) ... ok
> test_get_dev_byid_name_node_not_found (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_27_plus_disks_regression_issue (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_btrfs_in_partition (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_dell_perk_h710_md1220_36_disks (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_intel_bios_raid_data_disk (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_intel_bios_raid_sys_disk (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_luks_on_bcache (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_luks_sys_disk (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_mdraid_sys_disk (system.tests.test_osi.OSITests) ... ok
> test_scan_disks_nvme_sys_disk (system.tests.test_osi.OSITests) ... ok
> test_pkg_changelog (system.tests.test_pkg_mgmt.SystemPackageTests) ... ok
> test_pkg_latest_available (system.tests.test_pkg_mgmt.SystemPackageTests) ... ok
> test_pkg_update_check (system.tests.test_pkg_mgmt.SystemPackageTests) ... ok
> test_rpm_build_info (system.tests.test_pkg_mgmt.SystemPackageTests) ... ok
> test_zypper_repos_list (system.tests.test_pkg_mgmt.SystemPackageTests) ... ok
> test_get_con_config (system.tests.test_system_network.SystemNetworkTests) ... ok
> test_get_con_config_con_not_found (system.tests.test_system_network.SystemNetworkTests) ... ok
> test_get_con_config_exception (system.tests.test_system_network.SystemNetworkTests) ... ok
> test_get_dev_config (system.tests.test_system_network.SystemNetworkTests) ... ok
> test_get_dev_config_dev_not_found (system.tests.test_system_network.SystemNetworkTests) ... ok
> test_get_dev_config_exception (system.tests.test_system_network.SystemNetworkTests) ... ok
>
This will also help use to ensure a smooth migration for those moving from our last CentOS stable (3.9.2-57) to what will be our first Stable Rockstor 4 release.
But bar the AD stuff it does look to be shaping up quite nicely. Although the AD failure did highlight that we do need more wide spread testing which is exactly what we are now getting. So bar a few more ducks in row we are looking at an early 2021 release I think. All depends on developer time re the AD and shaping up the unit tests and tending to anything else we need to get sorted. @Flox has also submitted a much antisipated docker networking pr that I’d also like to get in before our initial 4 release, simply because it is so extensive and it would be challenging to leave it too long and could end up costing more time to get this in if we leave it much longer. So when I next get enough time to properly review this docker networking pr I’m hoping to pop this into testing for a bit with the hope we can get it out for our first 4 release later. That way we start as we mean to go on with that capability built in. And given it paves the way for tending to some other outstanding Rock-on short falls that seems to make sense. But again, all down to finding the quality time and but bit by bit and we should get there.
So in short we will have to have a little run in testing again first to address the above and then we can do our first official 4 release. Quite looking forward to this myself. It’s been a long journey and I’m eager to start our next one re the Python 3 and other technical debt stuff in our subsequent testing run.
Hope that helps and thank a ton for all your cross arch feedback, much appreciated.
2 Likes