2006-03-25
Revision History | ||
---|---|---|
Revision 1.3.4 | 2006-03-25 | kwd |
Revision 1.3.3 | 2006-02-11 | kwd |
Revision 1.3.2 | 2006-01-18 | kwd |
Revision 1.3.0 | 2005-08-15 | kwd |
Revision 1.2.6 | 2005-02-01 | kwd |
Revision 1.2.4 | 2004-12-28 | kwd |
Abstract
log4sh is a logging framework for shell scripts that works similar to the other wonderful logging products available from the Apache Software Foundataion (eg. log4j, log4perl). Although not as powerful as the others, it can make the task of adding advanced logging to shell scripts easier. It has much more power than just using simple "echo" commands throughout. In addition, it can be configured from a properties file so that scripts in a production environment do not need to be altered to change the amount of logging they produce.
Table of Contents
List of Tables
List of Examples
Table of Contents
Log4sh has been developed under the Bourne Again Shell (bash) on Linux, but great care has been taken to make sure it works under the default Bourne Shell of Solaris (sh) as this happens to be the primary platform used by myself.
Tested Operating Systems
Cygwin
FreeBSD
Linux
Solaris 8, 9, 10
Tested Shells
Bourne Shell (sh)
Bourne Again Shell (bash)
Korn Shell (ksh, pdksh)
A list of contributors to log4sh can be found in the source archive as doc/contributors.txt
. I want to personally thank all those who have contributed to make this a better tool.
Feedback is most certainly welcome for this document. Send your additions, comments and criticisms to the following email address: <kate.ward@forestent.com>
.
First things first. Go to the directory from which you extracted the log4sh software. In there, you should find a Makefile
file. If you find one, you are in the right place. We need to setup the environment for running tests, so from this directory, execute the make command as shown below. Once this is done, the test
directory will be prepared with everything needed to run the log4sh tests.
Prepare your environment.
$ make test-prep $ cd test
Example 2.1. Hello, World!
Ok. What kind of a quickstart would this be if the first example wasn't a "Hello, World!" example? Who knows, but this isn't one of those kind of quickstarts.
Run the Hello World test.
$ ./test-hello_world 0 [main] INFO shell - Hello, world
You should have seen output similar to that above. If not, make sure you are in the right location and such. If you really had problems, please send a letter to the log4sh maintainers. Who knows, maybe you already found a bug. Hopefully not!
The Hello World test is about as simple as it gets. If you take a look at the test, all it does is load log4sh, reset the default logging level from ERROR to INFO, and the logs a message. As you can see, it didn't take much to setup and use log4sh.
Example 2.2. Properties Configuration Test
In this example, a log4sh.properties
configuraiton file will be used to pre-configure log4sh before any logging messages are output. It demonstrates that a configuration file can be used to alter the behavior of log4sh without having to change any shell code.
Run the properties configuration test.
$ ./test-prop-config INFO - We are the Simpsons! INFO - Mmmmmm .... Chocolate. INFO - Homer likes chocolate ...
You should see much more output on your terminal that what was listed above. What is actually happening is log4sh is outputting information to STDERR using logging statements that were stored in the test-common
script. In addition, there were multiple logfiles generated (take a look in the directory), and output was written to syslog. Take a look at both the property configuration script and the common script if you like to see what is happening. What you will notice is that nowhere was it configured to write to the any of those different locations. The log4sh.properties
configuration file did all of that work. Take a look at it too. You might be amazed with how easy it was to write to so many locations with such a small amount of code.
Example 2.3. Runtime Configuration Test
This example is exactly like the last example as far as output is concerned (they both execute the same logging messages), but this one is configured instead at runtime with function calls. It demonstrates that log4sh is fully configurable at runtime.
Run the runtime configuration test.
$ ./test-runtime-config INFO - We are the Simpsons! INFO - Mmmmmm .... Chocolate. INFO - Homer likes chocolate ...
You should again see much more output on your terminal that what was listed above. The output should also have been exactly the same (except that the times were different) as the above example. This is because the same logging commands were used. If you take a look a look in the test-runtime-config
script though, you will see that log4sh was configured completly at runtime. The log4sh.properties
was not used. It shows that log4sh can be fully configured without a pre-existing configuration file. This isn't nearly as friendly as using the configuration file, but there are times when it is needed.
Table of Contents
The usage of log4sh is simple. There are only a few simple steps required to setup and use log4sh in your application.
preconfigure log4sh (properties file)
source the log4sh script code into the shell script
configure log4sh in code (optional)
call logging statements
To preconfigure log4sh, create a properties file (see the Properties File later in this document). If the properties file is not located in the same directory as log4sh, set the LOG4SH_CONFIGURATION
environment variable to the full path to the properties file. If you do not wish to preconfigure log4sh, please read the Configure log4sh in code section later in this chapter.
To source the code into your script (also known as including), one uses the sourcing ability of shell to source one script into another. See the following quick example for how easy this is done.
Example 3.1. Sourcing external shell code into current program
#! /bin/sh # source log4sh from current directory . ./log4sh
Here is some sample code that looks for log4sh in the same directory as the script is located, as well as the current directory. If log4sh could not be found, it exits with an error. If log4sh is found, it is loaded, along with the log4sh.properties
file in the current directory (see the following example). It then logs a message at the INFO
level to STDOUT.
Example 3.2. Hello, world (using properties file)
#! /bin/sh # # log4sh example: Hello, world # myDir=`dirname $0` # find and source log4sh if [ -r "$myDir/log4sh" ]; then log4shDir=$myDir elif [ -r "./log4sh" ]; then log4shDir=. else echo "fatal: could not find log4sh" >&2 exit 1 fi . $log4shDir/log4sh # say Hello to the world logger_info "Hello, world"
Here is the log4sh.properties
file for the previous example. Save it in the same directory you are running the above script from.
Example 3.3. Hello, world; properties file
# # log4sh example: Hello, world properties file # # Set root logger level to DEBUG and its only appender to A1 log4sh.rootLogger=INFO, A1 # A1 is set to be a ConsoleAppender. log4sh.appender.A1=ConsoleAppender # A1 uses a PatternLayout. log4sh.appender.A1.layout=PatternLayout log4sh.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
If log4sh was not preconfigured, the default configuration will be equivalent the config shown below. Please note: log4sh will complain if no configuration file was found. If you meant for the default configuration to be used, or you want to configure log4sh via code, make sure to define the LOG4SH_CONFIGURATION
with the value of "none".
log4sh.rootLogger=ERROR, stdout log4sh.appender.stdout=ConsoleAppender log4sh.appender.stdout.layout=PatternLayout log4sh.appender.stdout.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
To configure log4sh in code, simply call the appropriate functions in your code. The following code sample loads log4sh from the current directory, configures it for STDERR output, and the logs a message at the INFO
level.
Example 3.4. Hello, world (configured in code)
#! /bin/sh # # log4sh example: Hello, world # # source log4sh (disabling properties file warning) LOG4SH_CONFIGURATION='none' . ./log4sh # set the global logging level logger_setLevel INFO # close the default STDOUT appender, and add a new STDERR appender appender_close stdout logger_addAppender stderr appender_setType stderr FileAppender appender_file_setFile stderr STDERR # say Hello to the world logger_info 'Hello, world'
Table of Contents
Log4sh can be configured with a properties file that is separate from the actual script where the logging takes place. By default, log4sh looks for its properties file called log4sh.properties
in the current directory. If the file is located elsewhere or with a different name, log4sh can be configured by setting the LOG4SH_CONFIGURATION
environment variable (eg. LOG4SH_CONFIGURATION="/etc/log4sh.conf"
).
A log4sh.properties
file that is completly empty is sufficient to configure log4sh. There will be absolutely no output however (which might just be what is desired). Usually though, some output is desired, so there is at least a recommended minimum configuration file. An explaination of the file follows the example.
Example 4.1. Recommended minimum log4sh.properties file
log4sh.rootLogger=INFO, stdout log4sh.appender.stdout=ConsoleAppender
In the first line, the root logger is configured by setting the default logging level, and defining the name of an appender. In the second line, the stdout appender is defined as a ConsoleAppender
.
Table 4.1. Logging Levels (from most output to least)
Level | Definition |
---|---|
TRACE | The TRACE level has the lowest possible rank and is intended to turn on all logging. |
DEBUG | The DEBUG level designates fine-grained informational events that are most useful to debug an application. |
INFO | The INFO level designates informational messages that highlight the progress of the application at coarse-grained level. |
WARN | The WARN level designates potentially harmful situations. |
ERROR | The ERROR level designates error events that might still allow the application to continue running. |
FATAL | The FATAL level designates very severe error events that will presumably lead the application to abort. |
OFF | The OFF level has the highest possible rank and is intended to turn off logging. |
An appender name can be any alpha-numeric string containing no spaces.
An appender can be set to one of several different types.
Table 4.2. Appender Types
Type | Definition | Supported? |
---|---|---|
ConsoleAppender | output sent to console (STDOUT) | yes |
FileAppender | output sent to a file | yes |
DailyRollingFileAppender | output sent to a file that rolls over daily | partial; logs written, but not rotated |
RollingFileAppender | output sent to a file that rolls over by size | partial; works, but nees improvement |
SMTPAppender | output sent via email | parital; works, but needs improvement |
SyslogAppender | output sent to a remote syslog daemon | partial; only localhost supported |
An appender can take several different options.
Table 4.3. Appender Options
Option | Definition | Supported? |
---|---|---|
DatePattern | configure a pattern for the output filename | no (ignored) |
File | output filename (special filename of STDERR used for logging to STDERR) | yes |
MaxBackupIndex | number of old logfiles to keep | no (ignored) |
MaxFileSize | maximum size of old logfiles | no (ignored) |
Threshold | logging level of the appender | yes |
An appender can be configured with various Layouts to customize how the output looks.
Table 4.4. Layouts
Layout | Definition | Supported? |
---|---|---|
HTMLLayout | layout using HTML | no (same as SimpleLayout) |
SimpleLayout | a simple default layout ('%p - %m') | yes |
PatternLayout | a patterned layout (default: '%d %p - %m%n') | yes |
An layout has many different options to configure how it appears. These are known as patterns.
Example 4.6. Setting an appender's layout pattern
log4sh.appender.A1.layout.ConversionPattern=%d [%p] %c - %m%n
Table 4.5. Pattern Options
Option | Definition | Supported? |
---|---|---|
c | Used to output the category of logging request. As this is not applicable in shell, the conversion character will always returns 'shell'. | partial (fixed) |
d |
Used to output the date of the logging event. The date conversion specifier may be followed by a date format specifier enclosed between braces, but this specifier will be ignored. For example, The default format of the date returned is equavilant to the output of the Unix date command with a format of | yes |
F |
Used to output the file name where the logging request was issued. The default value is equavilent basename $0. | yes |
L | This option is for compatibility with log4j properties files. | no (ignored) |
m | Used to output the script supplied message associated with the logging event. | yes |
n | This option is for compatibility with log4j properties files. | no (ignored) |
p | Used to output the priority of the logging event. | yes |
r | Used to output the number of seconds elapsed since the start of the script until the creation of the logging event. | partial (bash, ksh) |
t |
Used to output the current executing thread. As shell doesn't actually support threads, this is simply a value that can be set that can be put into the messages.i The default value is 'main'. | yes |
x | This option is for compatibility with log4j properties files. | no (ignored) |
X | Used to output the MDC (mapped diagnostic context) associated with the thread that generated the logging event. The X conversion character must be followed by an environment variable name placed between braces, as in %X{clientNumber} where clientNumber is the name of the environment variable. The value in the MDC corresponding to the environment variable will be output. | no (ignored) |
% | The sequence %% outputs a single percent sign. | yes |
There are some environment variables that can be used to pre-configure log4sh, or to change some of its default behavior. These variables should be set before log4sh is sourced so that they are immediately available to log4sh.
Here is the full list of supported variables.
Table 4.6. log4sh environment variables
Variable | Usage |
---|---|
LOG4SH_CONFIGURATION |
This variable is used to tell log4sh what the name of (and possibly the full path to) the configuration (a.k.a properties) file that should be used to configure log4sh at the time log4sh is sourced. If the value 'none' is passed, than log4sh will expect to be configured at a later time via run-time configuration. |
LOG4SH_CONFIG_PREFIX |
This variable is used to tell log4sh what prefix it should use when parsing the configuration file. Normally, the default value is 'log4sh' (e.g. 'log4sh.rootLogger'), but the value can be redefined so that a configuration file from another logging frame work such as log4j can be read. |
Table of Contents
Table 5.1. Appenders
string
|
Gets the Layout of an Appender at the given array index type=`_appender_getLayoutByIndex 3` | |||||||||||||||
string
|
Gets the current logging Level of an Appender at the given array index type=`_appender_getLevelByIndex 3` | |||||||||||||||
string
|
Gets the name of the appender at the given position in the appender array. appenderName=`_appender_getNameByIndex 3` | |||||||||||||||
string
|
Gets the Pattern of an Appender at the specified array index pattern=`_appender_getPatternByIndex 3` | |||||||||||||||
string
|
Gets the Type of an Appender at the given array index type=`_appender_getTypeByIndex 3` | |||||||||||||||
string
|
Generate a logging message given a Pattern, priority, and message. All dates will be represented as ISO 8601 dates (YYYY-MM-DD HH:MM:SS). Note: the ' Example:
| |||||||||||||||
void |
(future) Sets the name of the appender at the given position in the appender array. _appender_setNameByIndex 3 "myAppender"` | |||||||||||||||
void |
Disable any further logging via an appender. Once closed, the appender can be reopened by setting it to any logging Level (e.g. INFO). appender_close myAppender | |||||||||||||||
boolean
|
Checks for the existance of a named appender exists=`appender_exists myAppender` | |||||||||||||||
string
|
(deprecated) Gets the Type of an Appender at the given array index type=`appender_getAppenderType 3` | |||||||||||||||
string
|
Gets the Layout of an Appender type=`appender_getLayout myAppender` | |||||||||||||||
string
|
Gets the current logging Level of an Appender type=`appender_getLevel myAppender` | |||||||||||||||
string
|
Gets the Pattern of an Appender pattern=`appender_getPattern myAppender` | |||||||||||||||
string
|
Gets the Type of an Appender type=`appender_getType myAppender` | |||||||||||||||
void |
(deprecated) Sets the Type of an Appender (e.g. FileAppender) appender_setAppenderType myAppender FileAppender | |||||||||||||||
void |
Sets the Layout of an Appender (e.g. PatternLayout) appender_setLayout myAppender PatternLayout | |||||||||||||||
void |
Sets the Level of an Appender (e.g. INFO) appender_setLevel myAppender INFO | |||||||||||||||
void |
Sets the Pattern of an Appender appender_setPattern myAppender '%d %p - %m%n' | |||||||||||||||
void |
Sets the Type of an Appender (e.g. FileAppender) appender_setType myAppender FileAppender | |||||||||||||||
void |
Add and initialize a new appender logger_addAppender $appender | |||||||||||||||
void |
Add and initialize a new appender with a specific PatternLayout logger_addAppenderWithPattern $appender '%d %p - %m%n' |
Table 5.2. FileAppender
string
|
Get the filename of a FileAppender at the given array index _appender_file_getFileByIndex 3 | ||||||||||
string
|
Get the filename of a FileAppender appender_file_getFile myAppender | ||||||||||
void |
Set the filename for a FileAppender (e.g. "STDERR" or "/var/log/log4sh.log") appender_file_setFile myAppender STDERR | ||||||||||
void |
(deprecated) Set the filename for a FileAppender (e.g. "STDERR" or "/var/log/log4sh.log") appender_setAppenderFile myAppender STDERR | ||||||||||
void/boolean |
(deprecated) Sets the email subject for an SMTP appender appender_setAppenderSubject myAppender "This is a test" | ||||||||||
void/boolean |
Sets the email subject for an SMTP appender appender_smtp_setSubject myAppender "This is a test" |
Table 5.4. Logger
void
|
The base logging command that logs a message to all defined appenders log DEBUG "This is a test message"` | ||||||||||
void
|
This is a helper function for logging a message at the DEBUG priority logger_debug "This is a debug message"` | ||||||||||
void
|
This is a helper function for logging a message at the ERROR priority logger_error "This is a error message"` | ||||||||||
void
|
This is a helper function for logging a message at the FATAL priority logger_fatal This is a fatal message` | ||||||||||
void
|
This is a helper function for logging a message at the INFO priority logger_info "This is a info message"` | ||||||||||
void
|
This is a helper function for logging a message at the TRACE priority logger_trace "This is a trace message"` | ||||||||||
void
|
This is a helper function for logging a message at the WARN priority logger_warn "This is a warn message"` |
Table 5.6. SMTPAppender
string
|
Get the email subject for the given appender subject=`_appender_smtp_getSubjectByIndex 3` | ||||||||||
string
|
Get the email to address for the given appender email=`_appender_smtp_getToByIndex 3` | ||||||||||
void |
(deprecated)Set the to address for the given appender appender_smtp_setTo myAppender user@example.com | ||||||||||
string
|
Get the email subject for the given appender subject=`appender_smtp_getSubject myAppender` | ||||||||||
string
|
Get the to address for the given appender email=`appender_smtp_getTo myAppender` | ||||||||||
void |
Set the to address for the given appender appender_smtp_setTo myAppender user@example.com |
Table 5.7. SyslogAppender
string
|
Get the syslog facility of the specified appender by index facility=`_appender_syslog_getFacilityByIndex 3` | ||||||||||
string
|
(deprecated) Get the syslog facility of the specified appender by index facility=`appender_getSyslogFacility 3` | ||||||||||
void
|
(deprecated) Set the syslog facility for the given appender appender_setSyslogFacility myAppender local4` | ||||||||||
void
|
Get the syslog facility for the given appender facility=`appender_syslog_getFacility myAppender` | ||||||||||
string
|
(deprecated) Get the syslog facility for the given appender facility=`appender_syslog_getFacilityByName myAppender` | ||||||||||
string
|
(deprecated) Get the syslog facility of the specified appender by index facility=`appender_syslog_getFacilityByIndex 3` | ||||||||||
string
|
TODO: Get the syslog host of the specified appender host=`appender_syslog_getHost myAppender` | ||||||||||
void
|
Set the syslog facility for the given appender appender_syslog_setFacility myAppender local4` | ||||||||||
void
|
TODO: Set the syslog host for the given appender appender_syslog_setHost myAppender localhost` |
Table 5.8. Threads
string
|
Gets the current thread name. threadName=`logger_getThreadName` | |||||
void |
Removes the topmost thread name from the stack. The next thread name on the stack is then placed in the __log4shThreadName variable. If the stack is empty, or has only one element left, then a warning is given that no more thread names can be popped from the stack. logger_popThreadName | |||||
void |
Sets the thread name (eg. the name of the script) and pushes the old on to a stack for later use. This thread name can be used with the '%t' conversion character within a PatternLayout. logger_pushThreadName "myThread" | |||||
void |
Sets the thread name (e.g. the name of the script). This thread name can be used with the '%t' conversion character within a PatternLayout. logger_setThreadName "myThread" |
Table 5.9. arrays
integer
|
Find the position of element in an array pos=`_log4sh_findArrayElement "$array" $element` | |||||||||||||||
string
|
Retrieve the element at the given position from an array element=`_log4sh_getArrayElement "$array" $position` | |||||||||||||||
integer
|
Get the length of an array length=`_log4sh_getArrayLength "$array"` | |||||||||||||||
string
|
Return the topmost element on a stack without removing the element. element=`_log4sh_peekStack "$array"` | |||||||||||||||
string[]
|
Remove the top-most element from a stack. This command takes a normal log4sh string array as input, but treats it as though it were a stack. newArray=`_log4sh_popStack "$array"` | |||||||||||||||
string
|
Add a new element to the top of a stack. This command takes a normal log4sh string array as input, but treats it as though it were a stack. newArray=`_log4sh_pushStack "$array" $element` | |||||||||||||||
string[]
|
Place an element at a given location in an array newArray=`_log4sh_setArrayElement "$array" $position $element` |
Table 5.10. miscellaneous
void |
This is a cleanup function to remove the temporary directory used by log4sh. It should only be called by log4sh itself when it is taking control of traps. If there was a previously defined trap for the given signal, log4sh will attempt to call the original trap handler as well so as not to break the parent script. _log4sh_cleanup EXIT | |||||
string
|
Converts an internally used level constant into its external tag equivalent tag=`_log4sh_level2tag 3` | |||||
string
|
Creates a secure temporary directory within which temporary files can be created. Honors the TMPDIR environment variable if it is set. tmpDir=`_log4sh_mktempDir` | |||||
integer
|
Converts an externally used level tag into its internal constant equivalent level=`_log4sh_tag2level WARN` | |||||
void |
This is a cleanup function to remove the temporary directory used by log4sh. It is provided for scripts who want to do log4sh cleanup work themselves rather than using the automated cleanup of log4sh that is invoked upon a normal exit of the script. log4sh_cleanup |
Table 5.11. properties
string
|
Takes a string (eg. "log4sh.appender.stderr.File") and returns the prefix of it (everything before the first '.' char). Normally used in parsing the log4sh configuration file. prefix=`_log4sh_getPropPrefix $property"` | ||||||||||
void |
Configures log4sh using an appender property configuration statement _log4sh_propAppender $property $value | ||||||||||
string
|
(future) Configures log4sh with a result=`_log4sh_propLogger $property $value` | ||||||||||
void |
Configures log4sh with a
The first option is the default logging level to set for all of the following appenders that will be created, and all following options are the names of appenders to create. The appender names must be unique. _log4sh_propRootLogger $value | ||||||||||
string
|
Strips the prefix off a property configuration command and returns the string. E.g. "log4sh.appender.stderr.File" becomes "appender.stderr.File". newProperty=`_log4sh_stripPropPrefix $property` | ||||||||||
void |
Reads a properties file and calls appropriate configuration functions. pos=`_log4sh_findArrayElement "$array" $element` |
The idea of log4sh is obviously not novel, but the availibility of such a powerful logging framework that is available in (nearly) pure shell is. Hopefully you will find it useful in one of your projects as well.
If you like what you see, or have any suggestions on improvements, please feel free to drop me an email at <kate.ward@forestent.com>
.
Log4sh is licensed under the GNU Lesser Public License. The contents and copyright of this document and all provided source code are owned by Kate Ward.