Monitoring Microsoft Windows Updates

Introduction

Monitoring WSUS updates on Microsoft Windows Server is critical to ensure you get alerted when your systems need to be patched. The process to update Windows Updates on high priority servers implies proper planning to ensure no post-installation problems. If we could trust Microsoft patches for 100 %, installing WSUS updates on a system would be done the moment a maintenance schedule could be created for this system. Unfortunately in my personal experience, WSUS updates are more a cause of problems instead of a solution. That’s why we prefer to not install them too fast, as you might experience major issues with your production systems or with the software that is running on it. A recent example, a colleague accidentally patched some production SharePoint servers, which prohibited the creation of new sitecollections and caused issues with some icons. The only solution was to restore a backup…

Ideally the updates would first need to get tested on QA systems. If the QA servers are running for some times without issues, the production systems can get patched. The above is one of the reasons I spent some time combining the best features from the available Windows Update plugins on the Nagios Exchange.
Such as Christian Kaufmann’s idea to cache the list of Windows Updates into a file. This results in a much lower performance impact of the plugin on the servers you are monitoring. If you have any experience with WSUS updates, you will have noticed that the ‘TrustedInstaller.exe” process which is a MS Windows system process that takes care of querying the WSUS server and installing updates if requested. 

The plugin will count all available WSUS updates and output the count in every possible state. However it will only alert in case a set number of days have passed since the last successful update was installed. By using this method, you can then define a policy and agree to patch all systems which had no updates for a certain time. You could use different policies for QA and PR (production) systems to prevent problems. 

WSUS

 

Details

Some things you need to know about Windows Updates. Microsoft saves the date of the ‘last successful update’ in the registry. The location of the String Value is:

This date however is saved in the Greenwich Mean Time (GMT) or the Coordinated Universal Time (UTC) format. My plugin will try to translate this time to the local time format with the help of a function called Get-LocalTime. This function uses the [System.TimeZoneInfo] .NET class which is only usable if you have .NET 3.5 or higher. So keep in mind the ‘Last Successful Update’ date is in UTC format for servers where .NET 3.5 or higher is not installed.

The plugin will also check this registry key:

And give a warning if the system has a required reboot pending.

PSWindowsUpdate

Starting from Windows 10, Microsoft apparently decided to no longer make use of the above registry key. The only way I found to retrieve the last successful update date and time is with the help of the PSWindowsUpdate module. So I added another argument which allows you to select a different method named ‘PSWindowsUpdate’ to retrieve the necessary information. Please not that the default method is still the original method, I called ‘UpdateSearcher”

In order for this method to work, you will need to install the PSWindowsUpdate module in this location: C:\Windows\System32\WindowsPowerShell\v1.0\Modules. If you are using Powershell 5 you can just do:

I’ve included the 1.5.1.11 and 1.5.2 version of the module in the GitHub repository. Or you can download it on the Microsoft Script Center Repository.

How to monitor your WSUS updates?

  1. Please note that the default DaysBeforeWarning and DaysBeforeCritical parameters are set to 120 and 150. Feel free to adjust them as required or pass them as an argument.
  2. Put the script in the NSClient++ scripts folder, preferably in a subfolder Powershell.
  3. In the nsclient.ini configuration file, define the script like this:
  4. Make a command in Nagios like this:
  5. Configure your service in Nagios. Make use of the above created command. Configure something similar like this as $ARG1$:
    QA servers =>

    PR servers =>

  6. If you want to make use of the new ‘PSWindowsUpdate’ method you will need to have an argument like this:

(Almost) Final words

So why did I create another pluging to check WSUS updates? Because I’m using a system which completely automates Windows Update installation with the help of Nagios XI and Rundeck. The existing plugins did not meet my requirements.

Please note that there are several known issues with WSUS on some operating systems. It’s recommended to always update to the latest ‘Windows Update Client’. Please check Windows 8.1 and Windows Server 2012 R2 update history for more information. More specific, when using WIndows Server 2012 R2, you will really want the following KB’s:

  • KB3172614 => “July 2016 update rollup for Windows 8.1 and Windows Server 2012 R2”
  • KB3179574 => “August 2016 update rollup for Windows 8.1 and Windows Server 2012 R2”
  • KB3185279 => “September 2016 update rollup for Windows 8.1 and Windows Server 2012 R2”

When you don’t have these update rollup’s, checking  for updates and updating your Windows 2012 R2 systems could go very slow. In our case an update check could take up to 40 minutes instead of 10 seconds. 

Let me know on the Nagios Exchange what you think of my plugin by rating it or submitting a review. Please also consider starring the project on GitHub.

Rundeck 2.10 – Ultimate Open Source Job scheduler

Rundeck Review

June 2016, Nagios announced they were stopping development on Nagios Reactor. So I had to start looking for a replacement. After playing with Foreman, Jenkins, Rundeck and Stackstorm, I decided the best solution for my needs was definitely Rundeck. In this Rundeck review, I’ll try to go into detail on some of the most useful Rundeck features I’ve been using over the last years.

Rundeck Review

Rundeck was definitely a hidden gem in the open source automation landscape, which has been dominated by configuration management oriented tools, such as Ansible, Chef, Puppet and Salt. But imho we don’t always need full configuration management. Usage of a job scheduler and orchestrator is in a lot of cases a more suitable option. And an added bonus is that Rundeck integrates with Ansible thanks to this plugin.

Rundeck is being very actively developed, meaning they regularely release new features. The nice thing is that they truly listen to their community, by allowing us to vote for popular features in a Trello board. Feel free to create an ccount and vote for the features you think deserve priority development time.

So what if you want professional support? Then you can opt into Rundeck Pro, which has some additional features and pro plugins available. Ok, I hope this Rundeck review helps you take a better informed decision on which automation platform to start using in your digital transformation.

Rundeck Projects and Jobs

Rundeck projects will contain definitions about nodes, as well as a set a jobs that reference these nodes. Using access control policies allows you to choose which teams have access to perform actions on jobs. Each node in the Rundeck project can be customized with tags, allowing you to target each kind of node rather than reference specific hosts names or IP addresses. All these Rundeck features allow you to create job libraries with useful scripts. Integrating The Rundeck access, job and exeecution logs into an Elastic stack gives you full visibility of what’s happening in your Rundeck server.

You can group Rundeck jobs in folders and subfolders. A collapsed view of all jobs in my DAF project:

 

Rundeck Security

Please note I’m just listing a few security related topics in this Rundeck review. Please refer to the official Rundeck documentation for all information you need to setup a secure Rundeck instance.

Active Directory integration

Active Directory integration is a basic requirement for any automation tool. Using Active Directory groups allows you to group users and assign specific permissions to them. Please refer to the official Rundeck documentation if you want more information how to configure this.

Agentless SSH based automation

A critical feature of any automation tool is a way to encrypt it’s traffic. As RunDeck uses SSH for executing commands on nodes, it already has a big advantage over other protocols. SSH is a secure protocol used as the primary means of connecting to Linux servers remotely. When you connect, you will be dropped into a shell session, which is a text-based interface where you can interact with your server. For the duration of your SSH session, any commands that you type into your local terminal are sent through an encrypted tunnel and executed on your server. Clients generally authenticate either using passwords (less secure and not recommended) or SSH keys, which are very secure.

SSL / HTPS

The RunDeck URL also needs to be protected, otherwise attackers could easily sniff your network and extract usernames, passwords, job options and more from api calls or logins. This procedure decribes the steps that need to be taken in order to configure SSL for your RunDeck server. I decided to create my ow version of the official documentation, but it’s only applicable to Microsoft .pfx certificates.

SSL

How to configure SSL for RunDeck?

  • Generate a .pfx server certificate with your private root ca
  • Copy the generated server certificate <servername>.pfx to /etc/rundeck/ssl
  • Create a keystore to hold the server certificate <servername>.pfx

  • Retrieve the alias from the <servername>.pfx file

  • Import the Certificate and Private Key into the Java keystore

  • Create a keystore for the CA certificate

  • Add the CA certificate to the CA keystore

  • Edit /etc/rundeck/ssl/ssl.properties and update all properties with their current values:

  • Edit /etc/rundeck/profile and uncomment:

  • Edit /etc/rundeck/rundeck-config.properties

  • Edit /etc/rundeck/framework.properties

  • Make sure port 4443 is opened in the firewall:

  • Restart the rundeckd daemon

  • Tail the RunDeck logs to make sure everything works fine:

Final words

I’d love to give a big thanks to the Rundeck developers for making Rundeck available to the public. I’m sorry if important stuff is missing in this (basic) Rundeck review, I’ll try to add more information over time. It’s also on my to do to open source my Elastic pipeline configurations, which enable analytics on the access, job and execution logs.

Monitoring WordPress Updates

Introduction

WordPress regularly releases updates with new features and bug fixes. It is advised to update asap, so monitoring your website for new WordPress updates is critical to ensure your website is fully patched against potential attacks. Automatic background updates were introduced in WordPress 3.7 in an effort to promote better security, and to streamline the update experience overall. By default, only minor releases – such as for maintenance and security purposes – and translation file updates are enabled on most sites. In special cases, plugins and themes may be updated.

In WordPress, there are four types of automatic background updates:

  1. Core updates
  2. Plugin updates
  3. Theme updates
  4. Translation file updates

Core Updates

Core updates are subdivided into three types:

  1. Core development updates, known as the “bleeding edge”
  2. Minor core updates, such as maintenance and security releases
  3. Major core release updates

By default, every site has automatic updates enabled for minor core releases and translation files. 

Update Configuration

Automatic updates can be configured using one of two methods: defining constants in wp-config.php, or adding filters using a Plugin.

Configuration via wp-config.php

Using wp-config.php, automatic updates can be disabled completely, and core updates can be disabled or configured based on update type.

Constant to Disable All WordPress Updates

The core developers made a conscious decision to enable automatic updates for minor releases and translation files out of the box. Going forward, this will be one of the best ways to guarantee your site stays up to date and secure and, as such, disabling these updates is strongly discouraged.

To completely disable all types of automatic updates, core or otherwise, add the following to your wp-config.php file:

Constant to Configure Core WordPress Updates

To enable automatic updates for major releases or development purposes, the place to start is with the WP_AUTO_UPDATE_CORE constant. Defining this constant one of three ways allows you to blanket-enable, or blanket-disable several types of core updates at once.

WP_AUTO_UPDATE_CORE can be defined with one of three values, each producing a different behavior:

  • Value of true – Development, minor, and major updates are all enabled
  • Value of false – Development, minor, and major updates are all disabled
  • Value of 'minor' – Minor updates are enabled, development, and major updates are disabled

Note that only sites already running a development version will receive development updates. For other sites, setting WP_AUTO_UPDATE_CORE to true will mean that it will only get minor and major updates.

For development sites, the default value of WP_AUTO_UPDATE_CORE is true. For other sites sites, the default value of WP_AUTO_UPDATE_CORE is minor.

How to check for WordPress Updates

  1.  Add the IP addresses of the Nagios servers that need access to the Allowed array in check_wordpress_updates.php”
  2. Put check_wordpress_updates.php in the root of you WordPress installation”
  3. Put check_wordpress_updates.sh in you Nagios plugin folder and call it from Nagios interface”

The result should look like this:

Wordpress updates plugin output

Thanks to Kong Jin Jie and hteske on whose scripts this is based.