SHIFT

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


Sidebar

Recently Changed Pages:

View All Pages


View All Tags


LinkedIn




WIKI Disclaimer: As with most other things on the Internet, the content on this wiki is not supported. It was contributed by me and is published “as is”. It has worked for me, and might work for you.
Also note that any view or statement expressed anywhere on this site are strictly mine and not the opinions or views of my employer.


Pages with comments

View All Comments

syslogesxi

Configure Syslog on ESXi 5.1

Configuring syslog on ESXi 5.1 is a two step plan. First configure the ESXi host to use the syslog server and then open the firewall to allow for outgoing syslog traffic.

Configure The Syslog Host

Select the host and go to the Configuration tab, then in the Software panel click the Advanced Settings button and in the pop-up window go to Syslog. Fill in the syslog server in the “Syslog.global.logHost” advanced setting:

udp://sysloghost.local:514

esxisyslog01.jpg

Configure The Firewall

To allow for outgoing syslog traffic go again to the Configuration tab and in the Software panel click the Security Profile. You'll see the current firewall settings and in the right the option to configure the Properties. In the pop-up window you can check the syslog checkbox and click OK:
esxisyslog02.jpg

Limiting the Amount of Logging

Now what you'll notice is that ESXi 5.1 logs enormous amounts of syslog, for my humble test hosts with about 30-40 VMs and not that much going on I could reach easily over 200000 logs a day, sometimes topping to 400000 log messages in one day. I wasn't prepared for that amount of data so I quickly decided I had to do something about that, and that led to some serious lessons:

  1. First of all, syslog implementation is not finished
  2. You can't configure it like you're used to
  3. Syslog greps into the normal logging data stream and is not a separate logging process
  4. You can only configure the log level for hostd and vpxa
  5. And even these don't work completely

Conclusion, it is a hard lesson and completely a manual task to limit the amount of logging, so let's start with the easiest setting.

Limiting the SysLog Logging Level for Hostd and Vpxa

You can limit the amount of logging by going into the advanced settings again, then go to Config → HostAgent → Log.
Set the following settings:

Config.HostAgent.log.level to warning (default = verbose)
Vpx.Vpxa.config.log.level to warning (default = verbose)

You've now set the syslog level to warning, but let's continue with editing the amount of logging for again these processes, the rhttpproxy and fdm.

Setting LogLevels on the ESXi Host

NOTE: CHANGING THE SETTINGS AND FILES BELOW IS NOT SUPPORTED. ONLY DO SO IF DIRECTED SO BY VMWARE

Setup a SSH session to the host and log in.

HOSTD

NOTE: CHANGING THE SETTINGS AND FILES BELOW IS NOT SUPPORTED. ONLY DO SO IF DIRECTED SO BY VMWARE

Edit the file below to setup logging for the hostd process:

/etc/vmware/hostd # vi config.xml

Original file log setings:

  <log>
    <directory>/var/log/vmware/</directory>
    <level>warning</level>
    <maxFileNum>8</maxFileNum>
    <maxFileSize>524288</maxFileSize>
    <name>hostd</name>
    <outputToConsole>false</outputToConsole>
    <outputToFiles>false</outputToFiles>
    <outputToSyslog>true</outputToSyslog>
    <syslog>
      <facility>local4</facility>
      <ident>Hostd</ident>
      <logHeaderFile>/var/run/vmware/hostdLogHeader.txt</logHeaderFile>
    </syslog>

Add or change the lines below:

  <log>
    <level>warning</level>
    <syslog>
    <level>warning</level>
    </syslog>

VPXA

NOTE: CHANGING THE SETTINGS AND FILES BELOW IS NOT SUPPORTED. ONLY DO SO IF DIRECTED SO BY VMWARE

Edit the file below to setup logging for the hostd process:

/etc/vmware/vpxa # vi vpxa.cfg

Original file log setings:

  <log>
    <level>warning</level>
    <maxFileNum>10</maxFileNum>
    <maxFileSize>1048576</maxFileSize>
    <memoryLevel>verbose</memoryLevel>
    <outputToConsole>false</outputToConsole>
    <outputToFiles>false</outputToFiles>
    <outputToSyslog>true</outputToSyslog>
    <syslog>
      <facility>local4</facility>
      <ident>Vpxa</ident>
      <logHeaderFile>/var/run/vmware/vpxaLogHeader.txt</logHeaderFile>
    </syslog>

Add or change the lines below:

  <log>
    <level>warning</level>
    <syslog>
      <level>warning</level>
    </syslog>

RHTTPPROXY

NOTE: CHANGING THE SETTINGS AND FILES BELOW IS NOT SUPPORTED. ONLY DO SO IF DIRECTED SO BY VMWARE

Edit the file below to setup logging for the hostd process:

/etc/vmware/rhttpproxy # vi config.xml

Original file log setings:

   <log>
      <!-- controls where rolling log files are stored -->
      <directory>/var/log/vmware/</directory>
 
      <!-- name of log file -->
      <name>rhttpproxy</name>
 
      <!-- controls whether logger sends its output to console also -->
      <outputToConsole>false</outputToConsole>
 
      <!-- If true, log to files on disk -->
      <outputToFiles>false</outputToFiles>
 
      <!-- default size(in bytes) of each log file before rolling over to next -->
      <maxFileSize>524288</maxFileSize>
 
      <!-- default number of log files to rotate amongst -->
      <maxFileNum>8</maxFileNum>
 
      <!-- default log level -->
      <level>verbose</level>
 
      <!-- If true, logs to syslog -->
      <outputToSyslog>true</outputToSyslog>
 
      <!-- syslog configuration.  Only used if outputToSyslog is true. -->
      <syslog>
         <!-- syslog identifier to use when logging -->
         <ident>Rhttpproxy</ident>
 
         <!-- syslog facility to use when logging -->
         <facility>local4</facility>
 
         <!-- The section header contents are placed in this file at startup.
              When vmsyslogd rotates the hostd log file, it logs the content of this
              file to syslog. This is being done so that we don't lose context on log
              rotations.
              IMPORTANT: Value needs to match that in onrotate entry in
              vmsyslog.d/hostd.conf
         -->
         <logHeaderFile>/var/run/vmware/rhttpproxyLogHeader.txt</logHeaderFile>
      </syslog>
   </log>

Add or change the lines below:

      <!-- default log level -->
      <level>warning</level>

FDM

NOTE: CHANGING THE SETTINGS AND FILES BELOW IS NOT SUPPORTED. ONLY DO SO IF DIRECTED SO BY VMWARE

Edit the file below to setup logging for the hostd process:

/etc/opt/vmware/fdm # vi fdm.cfg

Original file log setings:

  <log>
    <level>verbose</level>
    <outputToFiles>false</outputToFiles>
    <maxFileSize>0</maxFileSize>
    <maxFileNum>0</maxFileNum>
    <memoryLevel>trivia</memoryLevel>
    <outputToConsole>false</outputToConsole>
    <!-- If true, logs to syslog -->
    <outputToSyslog>true</outputToSyslog>
    <!-- syslog configuration.  Only used if outputToSyslog is true. -->
    <syslog>
      <!-- syslog identifier to use when logging -->
      <ident>Fdm</ident>
      <!-- syslog facility to use when logging -->
      <facility>local4</facility>
      <!-- The section header contents are placed in this file at startup.
           When vmsyslogd rotates the fdm log file, it logs the content of this
           file to syslog. This is being done so that we don't lose context on log
           rotations.
           IMPORTANT: Value needs to match that in onrotate entry in
           vmsyslog.conf.d/fdm.conf
       -->
      <logHeaderFile>/var/run/vmware/fdmLogHeader.txt</logHeaderFile>
    </syslog>
  </log>

Add or change the lines below:

<level>warning</level>

Add Comments

If required add comments like this, do not use “#” to comment, this will break the services:

    <!--
    20121109 Sjoerd Changed default level from verbose to warning
    -->

Restarting all Services

Don't forget to restart all services on the ESXi host and the best way to do is running this command:

services.sh restart
Note that the command will take about 5 minutes to complete and that your host will be unavailable in vCenter in the mean time. Just wait patiently for the server to come available again. This will have no impact on your VMs, unless you have automatic startup/shutdown enables which could be the case if you upgraded from previous versions.
I've seen HA kick in, although always failed to do anything. Maybe you could stop HA before you do this.

Filtering Messages

Even though the amount of messages should be drastically lowered, they still come with lot, in my case about 60000 a day which is still too much. I came across this message:

Unable to parse maxram/maxrampercpu/minrampercpu value:

According to this thread VMware support states:

This is a known cosmetic issue and can be safely ignored as there is no underlying issue with the license being used, it could be resolved in the next release update.

This means you probably want to filter this in your log collector. I have ManageEngine Eventlog Analyze on which you can create the filter like this:

  • Go to Settings
  • Go to Database Filters
  • Click on New Filter
    • Give a descriptive name
    • In the “Drop the logs containing:” fill in, without the quotes but including “:”:
      • “unable to parse maxrampercpu value:”
  • Click Next
  • Select the hosts where you want to enable the filter, which would preferably be a group including all your ESXi hosts
  • Click Finish

Now the amount of logging is reduced to about 5000-8000 messages a day, which is still a lot but a hell less then before. Of course you can create more filters when required, or reduce the log levels of more processes using the method described above.

Resources

You could leave a comment if you were logged in.
syslogesxi.txt · Last modified: 2021/09/24 00:25 (external edit)