Logback – Dynamic creation of log files

Some project have some log constraints that are difficult to implements with Log4j.

Logback is an alternative to log4j. It gives you more possibilities and it can easily implements your log policies.

    log1
Origami by Brian Chan

In this post, I present a simple of Logback configuration for a maven project. This configuration implements a dynamic log files generation.

Sample project description

This post example is inspired from a batch log policy I dealt with. The batch processes N files and generates two logs for each processed file. The first one is a summary of logs with no stack trace for functional users and the second one is a detailed log for programmers.

The picture bellow presents this case of study.

logback
Below the summary of log policies to implement:

  1. Logging errors without stacktrace.
  2. Dynamic file logging (switch to new log file).
  3. Read logging directory from application properties file.
  4. Create log files name with timestamp.

1. Add Maven dependency

First we need to add Slf4j and logback dependency to the project.

<dependency>
	<groupId>org.slf4j</groupId>
	<artifactId>slf4j-api</artifactId>
	<version>1.7.5</version>
</dependency>

<dependency>
	<groupId>ch.qos.logback</groupId>
	<artifactId>logback-classic</artifactId>
	<version>1.0.13</version>
</dependency>

<dependency>
	<groupId>ch.qos.logback</groupId>
	<artifactId>logback-core</artifactId>
	<version>1.0.13</version>
</dependency>

<dependency>
	<groupId>ch.qos.logback</groupId>
	<artifactId>logback-access</artifactId>
	<version>1.0.13</version>
</dependency>

<dependency>
	<groupId>org.slf4j</groupId>
	<artifactId>jcl-over-slf4j</artifactId>
	<version>1.7.5</version>
</dependency>

2. Create a costom layout Class

To customise logging layout, we create a class which extends LayoutBase and override the doLayout method.

public class CustomLogLayout extends LayoutBase<ILoggingEvent>{

	@Override
	public String doLayout(ILoggingEvent event) {
		StringBuffer sbuf = new StringBuffer(128);
	    sbuf.append(new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date(event.getTimeStamp())));
	    sbuf.append(" ");
	    sbuf.append(String.format("%-6s", event.getLevel()));
	    sbuf.append(" ");
	    sbuf.append(event.getFormattedMessage());
	    if((event.getLevel() == Level.ERROR || event.getLevel() == Level.WARN) && event.getThrowableProxy() !=null) {
	    	sbuf.append(" - ");
	    	sbuf.append(StringUtils.substringBefore(event.getThrowableProxy().getMessage(), IOUtils.LINE_SEPARATOR));
	    }
	    sbuf.append(CoreConstants.LINE_SEPARATOR);
	    return sbuf.toString();
	}

}

3. Logback configuration

In this step we create Logback.xml config file that will use the costomLayout we created in the previous step and will implements the logs policies.

  • Line 3 indicates the path to properties file where it will look for some values. In our case the path is a system property.
  • Line 5 specifies a key for the timestamp value.
  • STIF appender creates a custom log files with the CustomLogLayout class.
  • loggerFileName is a MDC (Mapped Diagnostic Contexts) property. Changing this value will generate a new file.
  • SIFT-TRACE appender creates a verbose log file and it uses logback.classic.PatternLayout.

<configuration>

	<property file="${appli.config.path}" />

	<timestamp key="bySecond" datePattern="yyyyMMddHHmmss" />

	<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">

		<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
			<level>INFO</level>
		</filter>
		<discriminator>
			<Key>loggerFileName</Key>
			<DefaultValue>unknown</DefaultValue>
		</discriminator>
		<sift>
			<appender name="FILE-${loggerFileName}" class="ch.qos.logback.core.FileAppender">
				<File>${log.dir}/${loggerFileName}_${bySecond}.log</File>
				<Append>false</Append>
				<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
					<layout class="com.mycompany.log.CustomLogLayout" />
				</encoder>
			</appender>
		</sift>
	</appender>

	<appender name="SIFT-TRACE" class="ch.qos.logback.classic.sift.SiftingAppender">
		<discriminator>
			<Key>loggerFileName</Key>
			<DefaultValue>unknown</DefaultValue>
		</discriminator>
		<sift>
			<appender name="TRACE-${loggerFileName}" class="ch.qos.logback.core.FileAppender">
				<File>${log.dir}/TRACE_${loggerFileName}_${bySecond}.log</File>
				<Append>false</Append>
				<layout class="ch.qos.logback.classic.PatternLayout">
					<Pattern>%d %-5level %logger{5} - %msg%n</Pattern>
				</layout>
			</appender>
		</sift>
	</appender>

	<logger name="org.springframework" level="ERROR" />
	
	<root level="DEBUG">
		<appender-ref ref="SIFT" />
		<appender-ref ref="SIFT-TRACE" />
	</root>

</configuration>

4. Test config

In this step we test the logback configuration.

setup() method set system property for the properties file path where logback will look for loging directory path.

The test simulates processing N files and at the end it verifies that we have 2xN log files.

public class LogTest {

	private static final int TEST_FILE_NUM = 5;

	private static final File TEST_DIR = new File(System.getProperty("java.io.tmpdir"), "logbak-test-dir");

	@Before
	public void setUp() throws IOException {
		System.setProperty("appli.config.path", ClassLoader.getSystemClassLoader().getResource("logback.properties").getPath());
		TEST_DIR.mkdirs();
		FileUtils.cleanDirectory(TEST_DIR);
	}


	@Test
	public void testLogbackFileConfig() throws InterruptedException {

		Logger logger = LoggerFactory.getLogger(LogTest.class);

		// assume SLF4J is bound to logback in the current environment
		LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
		JoranConfigurator configurator = new JoranConfigurator();
		configurator.setContext(context);
		context.reset();

		try {
			configurator.doConfigure(ClassLoader.getSystemClassLoader().getResource("logback.xml").getFile());
		} catch (JoranException je) {
			StatusPrinter.print(context);
			Assert.fail("Erreur de chargement des paramètre logback");
		}

		for (int i = 1; i <= TEST_FILE_NUM; i++) {
			MDC.put("loggerFileName", "file" + i);
			logger.info("log in File{}", i);
			logger.debug("log in File{}", i);
			logger.error("log in File" + i, new Exception("My test exception", new Exception("my cause of exception")));
		}

		Assert.assertEquals(TEST_FILE_NUM * 2, TEST_DIR.listFiles().length);
	}
	
}

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s