Deploying a standby Module Server
TanOS supports configuration and data sync from a primary Module Server appliance to a standby Module Server appliance. Data is copied to the standby appliance on demand or according to a schedule that you specify.
To protect data consistency, the scheduled TMS sync job stops
each solution module service, performs the synchronization, and restarts the service before stopping the next solution module service. Be sure to determine a TMS sync schedule that does not disrupt solution module processes. You might have to adjust scheduled activities for the solution modules accordingly.
Requirements and limitations
A redundant cluster deployment has the following requirements:
- Each Module Server must be installed on the same appliance model (size).
- Each Module Server must run the same software version, including build number (for example, each must have build number 7.3.314.3424).
- The two Module Servers must be able to connect to each other via a reliable Ethernet connection. The connection requires a minimum throughput of 1 Gbps and a maximum round-trip latency of 30 ms.
Before you begin
- Configure basic network, host, and user settings on both appliances. See Completing the initial setup (Tanium Cloud Appliance).For physical appliances, see Completing the initial setup (hardware appliances). For virtual appliances, see Completing the initial setup (virtual appliances).
- Specify the IP addresses of the redundant cluster interfaces when you configure the IPsec tunnel.
- Specify the IP addresses of the Tanium traffic interfaces when you configure the Module Server IP addresses.
- Make sure your network security administrator has configured security rules to allow communication on the TCP ports that the Tanium core platform components use. In addition to the ports used by individual Module Servers, a Module Server in a cluster sends and receives sync traffic over an IPsec connection. The network security rules must allow ESP (50/ip) and IKE (500/udp, 4500/udp).
- Install the primary Module Server and complete the steps to set up the remote Module Server configuration on both the Tanium Server and Module Server as described in Installing an individual Tanium Module Server.
- Install the standby Module Server but do not complete the steps to set up the remote Module Server configuration. You complete those steps only when making the standby Module Server active.
Set up the IPsec tunnel
IPsec is used to ensure end-to-end security between the two appliances.
- Start two SSH terminal sessions so you can copy and paste between them:
- First Module Server
- Second Module Server
- Sign into each of the Module Server appliances as a user with the tanadmin role and go to the IPsec menu:
- Enter A to go to the Appliance Configuration menu.
View screen------------------------------------------------------
>>> Appliance Configuration <<<
1: Hostname/DNS Configuration
2: Networking Configuration
3: NTP Configuration
4: Syslog Configuration
5: SNMP Configuration
6: Module File Share Configuration
7: Reset all NICs to DHCP (VM only)
A: Security
X: Advanced Configuration
H: Help
R: Return to previous menu
------------------------------------------------------------------------------------------------------------
>>> Appliance Configuration <<<
1: Hostname/DNS Configuration
2: Networking Configuration
3: NTP Configuration
4: Syslog Configuration
5: SNMP Configuration
6: Module File Share Configuration
A: Security
X: Advanced Configuration
H: Help
R: Return to previous menu
------------------------------------------------------ - Enter 2 to go to the Networking Configuration menu. View screen
------------------------------------------------------
>>> Appliance Configuration -> Networking <<<
1: Network Interfaces
2: IPSEC Configuration
3: Routing Configuration
4: Restart Networking
T: NIC Teaming
H: Help
R: Return to previous menu
------------------------------------------------------ ------------------------------------------------------
>>> Appliance Configuration -> Networking <<<
1: Network Interfaces
2: IPSEC Configuration
3: Routing Configuration
4: Restart Networking
H: Help
R: Return to previous menu
------------------------------------------------------ - Enter 2 to go to the IPSEC menu. View screen
------------------------------------------------------
>>> Appliance Configuration -> IP -> IPSEC <<<
1: Display Local IPSEC host key
2: Display IPSEC Configuration
3: Configure IPSEC
4: Delete IPSEC
5: Restart IPSEC
6: Test IPSEC
H: Help
R: Return to previous menu
------------------------------------------------------
- On the second appliance, copy the IPsec host key to the clipboard:
- From the IPSEC menu (A-2-2), enter 1 to view the local IPsec host key. View screen
>>> Appliance Configuration -> IP -> IPSEC -> Display Host Key <<<
Local Host Key Information:
0sAwEAAbnlzZ6venWVMdFLWEHGNEd6bMnNMVBkH+Ye3f7y360CbeBa6SSTOzI0NqHNOCnTWBDEMVWpfE3Dk/2feh1rjHpNp
MdhknhO5+8B47Q9HsH7DEGN4VoybNtH42xVKnApD51CGkH4Ns2o7JfHLUo+Dkv1Tw03b2vGNs/m//bLcUHwFKQLd1xKNOkR
4BhbQ0d7AVctY5tIKzhA8BS+aIkI7XuKCfy9YEMPOgyWVvPY2UTRXwvTcvK0+JHyxuu0UL6yvIVKeV1H8ohSbrDD213ut8l
DQJ6KbJQ5Zl/x3A0LrgTg8l0jNCGIB6d0oPiKpL7vePApViTTAgGh2l2b3KiwZldH6fof/h+dADWWuwcggek4NzyI1DRtBb
dA5bWuAEYdzC3038/N++FQnKJ5QRXe+b9O9aHf8VTUjZbFWc/5Q0wYeQ1GoTny1OiWArEDJUoW8IYUQIqPvmItG76zKwfIL
1z07dMIAs71W3L/X0QthogKCAYYawktpgukqJ3HHGkAWQFHgHKoJxVhJeF4Lw46G2OQaO1Eg5LHzYGS6BnEUG896yXvlxZJ
WZgG7Y9yj3mIea9ltV7/ifPIbXoMRkWMYbnK81biqw1j2yPJfj6MrHet5LrI+kUpA8FUCgY1tIdQPdXMU6F0K+wWapSzCtb
s8D343LYOMmiKgP2rZpFPOEc9
Press enter to continue
- Copy the key to the clipboard.
- On the first appliance, from the IPSEC menu, enter 3 and follow the prompts to configure the IPsec tunnel on the first appliance. When prompted, paste the IPsec host key for the second appliance. View screen
>>> Appliance Configuration -> IP -> IPSEC -> Configure IPSEC <<<
To configure IPSEC connectivity between TanOS systems the following information is required
- Local IP address
- Remote IP address
- Remote host key
Any exiting configuration will be overwritten!
Would you like to continue with IPSEC configuration? [Yes|No]: yes
Please provide the local ip address: 192.168.76.101
Please provide the remote ip address: 192.168.76.102
Please provide the remote hostkey: 0sAwEAAbnlzZ6venWVMdFLWEHGNEd6bMnNMVBkH+Ye3f7y360CbeBa6SSTO
zI0NqHNOCnTWBDEMVWpfE3Dk/2feh1rjHpNpMdhknhO5+8B47Q9HsH7DEGN4VoybNtH42xVKnApD51CGkH4Ns2o7JfHLUo+
Dkv1Tw03b2vGNs/m//bLcUHwFKQLd1xKNOkR4BhbQ0d7AVctY5tIKzhA8BS+aIkI7XuKCfy9YEMPOgyWVvPY2UTRXwvTcvK
0+JHyxuu0UL6yvIVKeV1H8ohSbrDD213ut8lDQJ6KbJQ5Zl/x3A0LrgTg8l0jNCGIB6d0oPiKpL7vePApViTTAgGh2l2b3K
iwZldH6fof/h+dADWWuwcggek4NzyI1DRtBbdA5bWuAEYdzC3038/N++FQnKJ5QRXe+b9O9aHf8VTUjZbFWc/5Q0wYeQ1Go
Tny1OiWArEDJUoW8IYUQIqPvmItG76zKwfIL1z07dMIAs71W3L/X0QthogKCAYYawktpgukqJ3HHGkAWQFHgHKoJxVhJeF4
Lw46G2OQaO1Eg5LHzYGS6BnEUG896yXvlxZJWZgG7Y9yj3mIea9ltV7/ifPIbXoMRkWMYbnK81biqw1j2yPJfj6MrHet5Lr
I+kUpA8FUCgY1tIdQPdXMU6F0K+wWapSzCtbs8D343LYOMmiKgP2rZpFPOEc9
-
On the first appliance, copy the IPsec host key to the clipboard:
- From the IPSEC menu, enter 1 to view the local IPsec host key.
- Copy the key to the clipboard.
- Go to the second appliance and complete the IPsec configuration:
- From the IPSEC menu, enter 3 and follow the prompts to configure the IPsec tunnel on the second appliance. When prompted, paste the IPsec host key for the first appliance.
- Enter 6 to test the connection from the second appliance. View screen
>>> Appliance Configuration -> IP -> IPSEC -> Test IPSEC <<<
Testing secured connectivity to the remote ip.
Secure connectivity to remote IP 192.168.76.101 is working
Press enter to continue
- Go back to the first appliance and enter 6 to test the connection.
Configure sync
- Sign into the standby Module Server appliance as a user with the tanadmin role and complete the following steps:
- Enter 2 to go to the Tanium Operations menu.
- Enter D to go to the Module Server Sync menu. View screen
------------------------------------------------------
>>> Tanium Operations -> Module Server Sync <<<
IPSEC: Ready
Remote: 192.168.76.103
Role: No module server sync configured
1: Show Detailed Status
2: Configure IPSEC
3: Configure TMS Sync
4: Sync Now (Source role only)
5: Schedule Sync (Source role only)
H: Help
R: Return to previous menu
------------------------------------------------------ - Enter 3 and follow the prompts to enable sync on the standby Module Server. View screen
>>> Tanium Operations -> Module Server Sync -> Configure Sync <<<
Secured connectivity to remote IP 192.168.76.103 is working
This module server is currently NOT configured to sync with 192.168.76.103.
Would you like to enable synchronisation? [Yes|No]: yes
Is this system the active module server (source) or the stand-by module server (target)?
Attention - please configure this stand-by module server first!
A: Active module server (source)
S: Stand-by module server (target)
Please select: s
Stand-by module server selected - attempting to enable sync...
Stopping and disabling module server service...
Enabling export...
Configuring firewall...
Updating environment...
Finished enabling module server sync with stand-by role.
Press enter to continue
- Sign into the primary Module Server appliance as a user with the tanadmin role and complete the following steps:
- Enter 2 to go to the Tanium Operations menu.
- Enter D to go to the Module Server Sync menu.
- Enter 3 and follow the prompts to enable sync on the primary Module Server. View screen
>>> Tanium Operations -> Module Server Sync -> Configure Sync <<<
Secured connectivity to remote IP 192.168.76.104 is working
This module server is currently NOT configured to sync with 192.168.76.104.
Would you like to enable synchronisation? [Yes|No]: yes
Is this system the active module server (source) or the stand-by module server (target)?
Attention - please configure this stand-by module server first!
A: Active module server (source)
S: Stand-by module server (target)
Please select: a
Active module server selected
Starting and enabling module server service...
Connecting to stand-by module server (this might take a while)...
Updating environment...
Finished enabling module server sync with active role.
Press enter to continue
Perform a manual sync
- Sign into the primary Module Server appliance as a user with the tanadmin role.
- Enter 2 to go to the Tanium Operations menu.
- Enter D to go to the Module Server Sync menu.
- Enter 4 to initiate sync. The sync job details are logged to the screen. View screen
------------------------------------------------------
>>> Tanium Operations -> Module Server Sync <<<
IPSEC: Ready
Remote: 192.168.76.104
Role: Active sync (source)
Last return code: No status file found
1: Show Detailed Status
2: Configure IPSEC
3: Configure TMS Sync
4: Sync Now (Source role only)
5: Schedule Sync (Source role only)
H: Help
R: Return to previous menu
------------------------------------------------------
TanOS Version: 1.6.3
TanOS_Shell Version: 1.6.3
Please select: 4
Secured connectivity to remote IP 192.168.76.104 is working
Thu May 16 20:52:14 UTC 2019 Synchronizing dir Tanium ModuleServer - Started
Thu May 16 20:52:15 UTC 2019 Synchronizing dir Tanium ModuleServer - 85 files updated. sent 121,565 bytes received 611 bytes 244,352.00 bytes/sec
Thu May 16 20:52:15 UTC 2019 Synchronizing dir Tanium ModuleServer - Completed
Press enter to continue
Schedule sync jobs
- Sign into the primary Module Server appliance as a user with the tanadmin role.
- Enter 2 to go to the Tanium Operations menu.
- Enter D to go to the Module Server Sync menu.
- Enter 5 to go to the Schedule TMS Sync menu. View screen
>>> Schedule TMS Sync <<<
Current time: Thu 2019-05-16 21:06:37 UTC
Active: Disabled
Day of Month | 1 | 00:00 UTC
Crontab formatted time string: 00 00 1 * *
Pending: Disabled
Day of Month | 1 | 00:00 UTC
Crontab formatted time string: 00 00 1 * *
1: Disable Schedule
2: Enable Schedule
4: Schedule by Day of Month
5: Schedule by Day of Week
6: Select Time of Day
7: Activate Schedule Settings
H: Help
R: Return to previous menu
------------------------------------------------------The top of the menu shows active and pending settings. The changes you make are pending until you use menu 7 to make them active.
- Use the menu to configure the schedule:
- Enter 1 or 2 to toggle the enabled/disabled status for the schedule.
- Enter 3 or 4 to set the schedule by days of the month or days in a week.
- A comma (,) indicates separate days. For example, 1,15.
- A hyphen (-) indicates contiguous days. For example, mon-fri.
- Specify days of the week with three-letter abbreviations: sun, mon, tue, wed, thu, fri, sat.
- Enter 6 to set the time of day.
- Enter 7 to make your changes active.
View detailed status for Module Server sync
The top of the Module Server Sync menu shows configuration status and the last return code for the sync job. You can use menu 1 to view detailed status.
- Sign into the Module Server appliance as a user with the tanadmin role.
- Enter 2 to go to the Tanium Operations menu.
- Enter D to go to the Module Server Sync menu.
- Enter 1 to view the status. View screen
>>> Tanium Operations -> Module Server Sync -> Check Status <<<
Secured connectivity to remote IP 192.168.76.104 is working
Active sync state (source)
Current time: Thu May 16 21:08:18 UTC 2019
Last sync log:
#### Starting module server sync ####
Start time: Thu May 16 20:52:13 UTC 2019
SYNC_ROLE 1 found
Thu May 16 20:52:14 UTC 2019 Synchronizing dir TaniumModuleServer - Started
Thu May 16 20:52:14 UTC 2019 CMD: sudo nohup rsync -aAvp --delete-after --stats --exclude services /opt/Tanium/TaniumModuleServer/ /opt/mounts/sync_tms/
Thu May 16 20:52:15 UTC 2019 rsync return code 0
Thu May 16 20:52:15 UTC 2019 Synchronizing dir TaniumModuleServer - 85 files updated. sent 121,565 bytes received 611 bytes 244,352.00 bytes/sec
Thu May 16 20:52:15 UTC 2019 Synchronizing dir TaniumModuleServer - Completed
End time: Thu May 16 20:52:15 UTC 2019
%%%% Ended module server sync %%%%%
Last return code: OK
Press enter to continue
Enable the standby Module Server
The Module Server service on the standby appliance is not enabled. In the event the primary Module Server appliance is taken out of service:
- Enable the Module Server service on the standby appliance.
- Reconfigure the Tanium Server(s) connection to the remote Module Server so that it uses the IP address and host name for the newly active Module Server.
Add the Module server to the Appliance Array
If you set up a Tanium cluster with an Appliance Array, add the standby Module Server to the array.
